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The  study  purpose  is  to  provide  a  critical 
appraisal  of  the  Amy's  test  and  evalua¬ 
tion  systeai  and  to  recomend  change  where 
appropriate. 


Testing  is  big  business  in  the  Amy.  Sane  300  million 
dollars  and  18,000  people  are  directly  chargeable  to  test  and 
evaluation,  excluding  troop  support. 

The  subject  has  received  extensive  attention  over  the  past 
few  years.  Considerable  change  has  occurred  in  the  system.  In 
view  of  the  reorganization  of  the  DA  staff,  the  formation  of  the 
Operational  Test  Agency,  and  the  creation  of  the  Forces  Command 
and  the  Training  and  Doctrine  Command,  an  appraisal  is  necessary 
to  validate  the  system  or  to  recommend  changes.  Through  an 
analysis  of  present  problems,  an  improved  system  will  be  developed 
if  necessary,  including  definitions,  procedures,  organization  and 
responsibilities. 

The  Army  tests  for  two  basic  reasons:  First,  as  an  integral 
part  of  the  materia,  acquisition  process,  and,  second,  as  a 
critical  way  to  assess  innovative  ideas  in  organizations,  tactics, 
and  doctrine.  A  test  is  a  process  by  which  data  are  accumulated 
to  assist  in  determining  that  a  system  meets,  exceeds  or  fails  to 
meet  the  characteristics  ascribed  to  it. 
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TO  EXAMINE  THE  ARMV 
TEST  AND  EVALUATION 
SVSTEM. 


•  DETERMINE  PROBLEMS. 

•  DEVELOP  A  'BEST* 
SYSTEM. 

•  DETERMINE 

ASSIGNMENT  OF 
RESPONSIBILITIES. 
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The  group  followed  three  paths:  reviewed 
previous  studies  and  regulations,  inter¬ 
viewed  key  individuals,  analyzed  test  re¬ 
ports,  and  drew  on  their  own  experience. 


To  accomplish  the  purpose  of  the  study,  the  three-man  group 
reviewed  all  major  studies  of  test  and  evaluation,  the  most 
significant  ones  being  the  reports  of  the  President's  Blue  Ribbon 
Panel,  the  Priority  Project  Four  Group,  and  the  Army  Materiel 
Acquisition  Review  Committee.  All  regulations  were  analyzed 
in-depth,  the  most  significant  being  DOD  Directive  5000. 3 >  the 
DA  LOI  that  implemented  material  acquisition  guidelines:  and 
AR  10-4. 

Discussions  were  held  with  all  key  personnel  involved  in 
testing,  including  LTG  Starbird,  (Ret.),  DDRE;  GEN  DePuy,  TRADOC; 
LTG  Deane,  DCSRDA;  MG  McChrystal,  MASSTER;  MG  Brown,  TECOM;  and 
MG  Ochs,  OTEA.  All  shared  their  philosophy  of  testing  with  the 
study  group  and  injected  views  on  the  strengths  and  weaknesses  of 
the  present  system. 

The  study  is  limited  to  the  Army  system,  although  some  of  the 
report  contents  may  be  of  use  to  the  other  Services.  It  excludes 
the  testing  activities  of  special  elements  such  as  the  Surgeon 
General,  the  Chief  of  Engineers  and  the  Army  Security  Agency. 
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•  REVIEW  OF  PREVIOUS 
STUDIES  AND 
REGULATIONS. 

•  DISCUSSIONS  WITH 
KEV  PERSONNEL. 


REVIEW  OF  TEST 


Hwrjrv?. 
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Current  Army  Test  and  Evaluation  policies 
stem  from  DOD  guidance,  DA  regulations, 
letters  of  instructions,  and  directives. 


Since  1971,  DO)  has  emphasized  the  necessity  of  improving 
operational  testing  in  the  Services.  In  January  1973,  DOD  Direc¬ 
tive  5000.3  was  published  as  guidance  on  Test  and  Evaluation. 

The  major  thrust  of  the  directive  was  to  require  each  Service  to 
have  a  separate  agency  responsible  for  operational  testing  which 
would  report  its  results  directly  to  the  Chief  of  Service. 

The  Army  implemented  the  DOD  philosophy  by  publishing  a 
Letter  of  Instructions  (LOI)  incorporating  new  basic  policies 
for  materiel  acquisition.  Included  in  these  policies  was  the 
requirement  that  testing  be  divided  into  the  categories  of  Develop¬ 
ment,  Operational,  and  Force  Development  Tests. 

Since  publication  of  the  basic  policy,  the  Army  has  been  very 
slow  in  providing  further  guidance  on  testing.  In  January  1974, 

DA  issued  AR  10-4  which  provided  the  mission,  functions,  and  command 
relationships  of  the  Army's  separate  test  agency,  the  Operational 
Test  and  Evaluation  Agency.  The  remaining  regulations  have  been 
under  revision  since  1972  and  with  the  last  DA  staff  reorganization, 
even  the  latest  drafts  are  nullified. 
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ARWV  GUIDANCE, 
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LOI«  23  AUGUST  1972 
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ARMV  TEST 

REGULATIONS. 
AR  104 

RR  10-10  (DRAFT) 
AR  TI-3  (DRAFT) 
AR  71  -8 
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The  current  family  of  Army  tests  are  DT, 
OT  and  FDTE. 


As  part  of  its  guidance,  the  LOI  established  the  three  types 
of  tests: 

Development  Testing  is  planned,  conducted  and  evaluated  by 
the  developer  and  started  as  early  in  the  development  cycle  as 
possible.  It  should  first  test  components,  then  subsystems  and 
finally  prototypes  or  preproduction  models  of  the  whole  system. 

The  characteristics  of  DT  are  as  shown  on  the  right. 

Operational  Testing  is  planned,  conducted  and  evaluated  by 
either  the  designated  user  or  an  independent  test  agency.  It  is 
accomplished  in  as  realistic  an  operational  environment  as  possible. 
Characteristics  of  OT  are  shown  on  the  right. 

Force  Development  Test  and  Experimentation  consists  of  tests 
that  are  performed  to  support  the  force  development  process  by 
examining  the  impact,  potential  and  effectiveness  of  selected 
concepts,  doctrine,  tactics,  organization  and  materiel.  Ihey 
support  the  materiel  acquisition  process  by: 

•  assisting  in  the  development  of  HOC s 

•  assisting  in  understanding  the  total  performance  of  a 
materiel  system 

•  assisting  in  the  development  of  counter-countermeasures 
for  a  deployed  system. 
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TEST 

CHARACTERIST I CS 

DEVELOPMENT  TESTING  (plT] 

ASSESS  TECHNICAL  RISKS. 

DEMONSTRATE  THAT  ENGINEERING  DESIGN 
IS  COMPLETE  AND  ACCEPTABLE. 

DETERMINE  IF  DESIGN  RISKS  ELIMINATED. 


OPERATIONAL  TESTING  (OT) 


DETERMINE  MILITARY  POTENTIAL,  UTILITY, 
OPERATIONAL  EFFECTIVENESS,  AND 
OPERATIONAL  SUITABILITY. 

DETERMINE  FROM  USER  VIEW  IF  NEW 
SYSTEM  IS  BETTER  THAN  EXISTING 
(BENEFITS  VS  BURDENS). 

DETERMINE  NEED  FOR  MODIFICATION. 

DETERMINE  ADEQUACY  OF  ORGANIZATION, 
DOCTRINE,  OPERATIONAL  TECHNIQUES 
AND  TACTICS  FOR  SYSTEM  EMPLOYMENT 
AND  MAINTENANCE  SUPPORT. 


FORCE  DEVELOPMENT  TEST  AND  EXPERIMENTATION  (FDTE) 


-  ASSESSES  INTERDEPENDENCE  AMONG  DOCTRINE, 
TACTICS,  ORGANIZATION  AND  MATERIEL. 


SUPPORTS  MATERIEL  ACQUISITION  PROCESS. 
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The  Army  has  incorporated  high-level 
decisionmaking  in  its  materiel  acqui¬ 
sition  policies. 


As  a  part  o.t  its  new  materiel  acquisition  procedures,  adopted 
in  1972,  the  Army  established  an  Army  Systems  Acquisition  Review 
Committee  (ASARC)  in  parallel  with  the  Defense  Systems  Acquisition 
Review  Committee.  This  group  of  senior  officials  is  the  approving 
authority  for  all  major  systems  undergoing  development.  Major  systems 
include  all  materiel  developments  that  exceed  the  dollar  thresholds 
shown  on  the  right. 

Approval  authority  for  all  other  systems  is  by  means  of 
In-.'rocess  Review  (IPR),  normally  chaired  by  the  materiel  developer. 
The  Army  staff  has  designated  certain  systems,  classified  as  selected 
non-major  systems,  to  be  approved  by  Headquarters,  DA.  The  criteria 
for  these  systems  is  not  well  defined,  but  includes  those  develop¬ 
ment  projects  that  have  congressional  interest,  are  of  particular 
importance  to  the  Army,  or  are  of  sufficient  magnitude  to  warrant 
special  attention.  All  other  developmental  projects  are  merely 
classified  as  other  non-major  systems. 

Each  materiel  development  passes  through  four  milestones  during 
its  acquisition  cycle.  The  decision  for  a  system  to  pass  from  one 
milestone  to  another  is  made  by  the  appropriate  approval  authority, 
either  the  ASARC  or  IPR.  DT  and  OT  are  scheduled  to  coincide  with 
these  decision  meetings.  Since  there  are  three  ASARCs  (I,  II,  III) 
required  for  each  major  system  development,  regulations  also  provide 
for  the  same  number  of  DTs  and  OTs. 


10 


LEVELS  OF  DECISION 


TVPE 

ACQ. 

(APPROWIL 

LEVEL) 

DECISION 

eoov 

DOLLAR  REMARKS 

THRESHOLD 

(MILLION) 

i 

MAJOR 

OSARC 

ROTE  (90*)  SEC.ARMV 

3V3TEM 

(SECOEF) 

SELECTED 

ASARC 

PEMA(EOO*)  O*1  WW 

ASARC ID 
APPROVE 
OTHER  SOS. 

NON-MAJOR 

SVSTEMS 

(CRD) 

IPR 

RDTE(O-SO)  SELECTED 
PEMA(0-200)  0VACSFOR 

ORCRO 

OTHER 

nonmajor 

SVSTEMS 

(MATERIEL 

developer) 

IPR 

ROTE  >30)  NO  DA 
PEMA (o-aoo)  PARTICI¬ 
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Developmental  Tests  for  the  Amy  are  con¬ 
ducted  by  the  Test  and  Evaluation  Command 
of  the  Army  Materiel  Command. 


the  Deputy  Director  of  Defense  Research  and  Engineering  (Test 
and  Evaluation)  monitors  closely  all  major  acquisition  programs  of 
the  Department  of  Defense. 

The  Department  of  the  Array  staff  representative  for  DT  is  the 
Office  of  the  Deputy  Chief  of  Staff,  Research,  Development  and 
Acquisition  (DCSRDA)-- formally  OCRD.  Here  a  10-c  an  Test  and  Evalua¬ 
tion  Branch  manages  the  DT  program. 

The  Test  and  Evaluation  Command  (TECOM)  is  one  of  nine 
subordinate  elements  of  the  Army  Materiel  Comaand  (AMC).  TECOM 
provides  independent  tes^  and  evaluation  for  AMC,  its  commodity 
commands,  private  industry  and  other  Government  agencies. 

The  14  Installations  of  TECOM  perform  the  Developmental  Tests 
of  new  material.  The  Engineering  Test  portion  of  DT  is  usually 
conducted  at  one  of  the  six  proving  Grounds  and  the  two  Environ¬ 
mental  Test  Centers. 

The  Service  Test  portion  of  DT  is  usually  conducted  at  the 
six  Branch  Boards  which  are  co-located  with  the  appropriate  schools. 

Consisting  of  some  14,000  personnel,  TECOM  contains  the  bulk 
of  the  dedicated  Array  test  and  evaluation  force. 
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Operational  Tests  for  the  Army  are 
managed  by  OTEA  and  TRADOC. 


The  Deputy  Director  Defense  Research  and  Engineering  (Test 
and  Evaluation)  provides  galley  guidance  and  establishes  procedures 
for  Operational  Testing.  As  a  result  of  the  criticism  directed  at 
OT  by  the  Blue  Ribbon  Defense  Panel  in  1970,  the  Department  of  Defense 
has  been  extremely  active  in  this  field. 

Each  Service  has  been  directed  to  establish  an  independent  test 
agency  to  plan,  direct  and  evaluate  OT,  with  the  results  being  reported 
directly  to  the  service  chief. 

The  Assistant  Chief  of  Staff  for  Force  Development  (ACSFOR) 
managed  OT,  through  OTEA,  prior  to  the  recent  reorganization  of  the 
DA  staff.  With  the  elimination  of  ACSFOR,  OTEA  was  placed  directly 
under  control  of  the  Army  Chief  of  Staff. 

Major  and  selected  non-major  operational  testing  is  actually 
executed  by  troop  units  within  Forces  Command  or  an  overseas  command 
under  the  direction  of  OTEA.  There  are  currently  some  51  critical 
systems  (major  and  selected  non-major)  in  the  process  of  test  planning, 
direction  or  evaluation  by  OTEA. 

All  OT  other  than  that  selected  cor  intensive  OTEA  management 
is  conducted  by  TRADOC.  There  are  currently  160  such  systems. 


OPERATIONAL  TESTING 

(  LINES  OF  COMMUNICATION  ) 


OTHER 

NON-MAJOR  MAJOR  SWIEMS  -  24- 
(160)  SELECTED  NON-MAJOR-27 


Force  Development  test  and  experimentation  I 
for  the  Army  Is  primarily  conducted  by  I 
HAMSTER  and  CDEC.  I 

ACSFOR  provided  DA  staff  management  of  FDTE  until  the  majority 
of  the  function  was  passed  to  OTEA,  its  field  agency.  Although  the 
Deputy  Chief  of  Staff  for  Military  Operations  (DCS OPS)  assumed  many 
ACSFOR  functions  during  the  DA  staff  reorganization,  its  participa¬ 
tion  in  FDTE  is  currently  being  debated. 

There  are  presently  two  test  agencies  concentrating  on  FDTE— 
the  Modern  Army  Selected  System  Test,  Evaluation,  and  Review  (MASSTER) 
at  Fort  Hood,  under  FORSCOM,  and  the  Combat  Developments  Experimen¬ 
tation  Command  (CDEC)  at  Fort  Ord,  under  TRADOC. 

Broadly  speaking,  MASSTER  conducts  large-unit  subjective  FDTE 
utilizing  the  two  FORSCOM  armored  divisions  which  report  to  the 
MASSTER/ III  Corps  Commander.  Extensive  instrumentation  now  being 
procured  will  substantially  enhance  MASSTER's  capability. 

CDEC  conducts  small-unit  objective  tests  (experiments)  utiliz¬ 
ing  organic  troops  of  its  1900-man  Experimentation  Group.  CDEC 
is  supported  by  a  contract  scientific  support  laboratory  of  approxi¬ 
mately  100. 

FDTE  requests  are  normally  generated  by  the  schools  and  functional 
centers,  then  forwarded  to  TRADOC  for  assignment  through  OTEA  to  either 
CDEC  or  MASSTER,  as  appropriate. 

Although  DOD  could  become  active  in  directing  the  Army's  Force 
Development  Test  and  Experimentation  (FDTE)  program,  it  has  not  been 
so  to  date. 
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The  total  Army  testing 
requirement  is  immense. 


Though  predicting  the  foreseeable  Army  test  requirements  is 
difficult,  it  is  possible  nevertheless  to  estimate  an  average  yearly 
test  load  from  historical  evidence  and  current  planning  documents. 

Considering  the  "user"  test  side  of  the  coin  first,  over  the 
next  five  years  there  are  scheduled  operational  tests  for  24  major 
systems  (such  as  tanks,  helicopters,  and  air  defense  weapons)  and 
27  selected  non-major  systems  (such  as  mortars,  drones  and  radars). 
Though  small  in  number,  these  include  all  the  major  material  the 
Amy  plans  to  acquire.  Moreover,  160  "other"  systems  require  opera¬ 
tional  testing  also.  To  assist  the  combat  development  process, 
nearly  70  tests  of  tactics,  doctrine  and  organization  are  scheduled. 
Some  300  pending  ROC's  may  prove  grist  for  this  test  mill,  as  well. 

On  the  development  side  of  the  coin,  by  far  the  bulk  of  Army 
testing  is  done  for  the  material  developer,  AMC.  Test  requirements 
ranging  from  customer  assistance  to  DT's  of  major  systems  run  the  AMC 
load  to  an  estimated  1200  yearly. 

A  reasonable  estimate  of  the  yearly  Army  test  requirement  is 
125  "user"  and  1200  AMC  tests. 
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MAGNITUDE  OF 
ARMV  TESTING 


Currentlv-  programmed  tests 

FOR  1974-1978. 

Major  24  (fvtp) 

Selected  Non- major  27  (fvtp) 
Other  non- major  i&o 

Force  development  68  (fvtp) 

TEST  ANO  EXPER, 


Estimated  annual  test  load. 

Tecom  tests  1200 

l)3ER  TESTS  125 
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FINDING  AND  RECOMMENDATION 


Test  goals  are  often  mixed  with  test  methods, 
one  being  defined  in  terms  of  die  other. 

This  leads  to  confusion  and  misunderstand¬ 
ing  and  is  due,  in  part,  to  unclear  regula¬ 
tions  and,  in  part,  to  lack  of  understand¬ 
ing  of  the  philosophy  of  testing.  AR  71-8 
should  be  rewritten  to  include  separate  and 
distinct  'goals'  and  'methods'  sections  as 
spelled  out  in  Chapter  VII. 

It  became  apparent  during  the  study  that  confusion  existed  con¬ 
cerning  test  goals  and  test  methods.  Existing  AR's  lump  the  two 
together  saying,  for  example,  "...  Force  Development  Tests  are  tests 
ranging  from  small  ...  highly  instrumented  experiments  to  broad  ... 
highly  subjective  field  tests  that  are  performed  to  support  the  force 
development  process...."  The  definitions  are  too  broad  to  be  useful 
to  the  planner  and  they  do  not  clarify  the  complex  philosophical 
nature  of  testing. 

Fundamental  goals  of  Army  testing  are  as  shown  on  the  right. 
There  is  a  continuum  of  test  methods  that  may  be  used  to  satisfy 
the  goals.  The  method  selected  will  ultimately  depend  on  a  number 
of  factors  including  information  needs  and  resource  constraints. 

Across  the  method  continuum  are  two  extremes:  the  field 
experiment  (highly  objective)  and  the  field  evaluation  (highly 
subjective).  In  the  middle  ground  is  a  method  that  combines  aspects 
of  each,  the  field  trial.  Eight  defining  characteristics  for  each 
test  method  have  been  developed  in  Chapter  VII  and  illustrate  the 
point  more  completely.  Although  effort  should  always  be  made  to 
strive  for  complete  objectivity  in  any  test,  some  questions  may 
only  be  addressed  through  subjective  field  evaluation. 
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CAUSING  CONFUSION* 
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GOALS 

MATERIEL  PERFORMANCE  (DT) 

MATERIEL-  SOLDIER- MISSION 
PERFORMANCE  (OTJ 

MATERIEL  CONCEPT  VAUOITV  (FT-M) 

ORGANIZATION  -  TACTICS  -  DOCTRINE 
VAUOITV  (FT-3) 


METHODS 


OBJECTIVITY 

» . 

FIELD 

EXPERIMENT 


Field  field 

TRIAL  EVALUATION 


INCLUDE.  SEPARATE.  GOALS 
AND  MEIMOOS  SECTIONS 
IN  APPROPRIATE  REGULATIONS 


FINDING  AND  KECOMCNDATION 


FDTE  is  not  receiving  sufficient  emphasis 
due  to  vague  regulations  and  loose  defini¬ 
tions.  By  separating  Force  testing  into 
two  types  of  tests,  FT-M  and  FT-S,  the  FT 
system  will  be  more  responsive.  New  defi¬ 
nitions  are  proposed  in  Chapter  VII  and  the 
recotsnended  system  appears  on  the  right. 


History  provides  ample  proof  of  the  value  of  force  testing. 

Previously,  such  testing  was  done  urder  a  variety  of  terms  such 

as  Troop  Test,  Field  Test,  Field  Evaluation  and  Field  Experiment. 

More  recently,  the  term  FDTE  has  been  developed  to  cover  all  of 

this  type  of  testing.  The  present  definition,  however,  is  too 

vague  to  be  of  much  utility.  It  is  our  view  that  force  tests 

should  be  separated  into  two  categories: 

Force  Test- -Material  (FT-M)  aids  the  material 
acquisition  process  by  providing  information 
toward  the  establishment  or  validation  of  a  ROC. 

Force  Test- -Structure  (FT-S)  examines  the  valid¬ 
ity  of  proposed  tactics,  doctrine  and  organization. 

The  top  figure  on  the  right  illustrates  the  proposed  system. 
The  outer  blocks  represent  the  entire  test  while  the  inner  blocks 
represent  the  relative  importance  of  the  material  to  the  test. 

The  relationship  of  the  four  tests  to  one  another,  with  possible 
test  outcomes,  is  illustrated  by  the  figure  on  the  bottom. 

It  is  not  our  intent  that  every  ROC  proposal  or  every  question 
involving  organization,  tactics  and  doctrine  be  subject  to  this 
system--only  those  ideas  that  are  amenable  to  the  test  process. 
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FINDING  AND  RECOMMENDATION 


In  implementing  DOD  policy  ,  the  Any  has  not 
maintained  a  flexible  attitude  toward  testing 
and  has  overstructured  Its  OT  procedures  and 
overeaphaslzed  organizational  placement  of 
the  operational  tester.  To  resolve  these 
problems,  two  actions  are  required.  First, 
publish  AR  70-10  Inedlately  and  In  it 
specify  that  there  not  be  a  rigid  requirement 
for  all  phases  of  OT.  Secondly,  give  further 
study  to  the  organizational  placeaent  of  OTEA. 


It  has  been  two  years  since  the  Army  changed  its  testing  pro¬ 
cedures,  and  18  aonths  since  DOD  published  its  guidance  on  testing; 
however,  implementing  regulations  (AR  70-10  and  AR  71-3)  are  still 
in  draft  form.  The  latest  draft  of  AR  70-10  appears  to  be  vague 
and  controdictory.  Even  though  the  various  developmental  systems 
are  categorized  as  major,  selected  non-major,  and  other  non-major 
no  such  differentiation  is  made  for  testing.  In  fact,  the  vagueness 
of  the  draft  is  such  that  all  phases  of  both  0T  and  DT  could  be 
expected  for  all  developments,  regardless  of  their  relative  Importance. 

Although  DOD  guidance  directs  that  both  development  and  opera¬ 
tional  test  results  will  be  reported  to  the  Military  Service  Chief, 
it  is  not  evident  that  this  would  require  the  testers  to  be  located 
at  that  echelon.  It  is  our  opinion  that  the  intent  of  the  DOD 
directive  is  that  neither  the  development  tester  nor  the  operational 
Lester  need  be  placed  directly  under  the  Office  of  the  Chief  of 
Staff . 
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FINDING  AND  RECOMMENDATION 


The  Coordinated  Test  Program  is  not  being 
properly  coordinated  in  order  to  insure  that 
(1)  the  right  issues  are  tested  and  (2)  there 
is  no  duplication  of  testing  between  DT  and  OT. 

To  accomplish  this,  the  DCSRDA  should  have  the 
overall  responsibility  for  the  management  of 
the  CTP. 

The  draft  regulations  governing  the  CTP  are  clear  as  to 
intent  of  coordination  and  specify  that  the  operational  tester 
will  concur  in  the  plan.  The  CTP  has  the  making  of  a  fine 
management  tool;  however,  its  preparation,  coordination,  distri¬ 
bution  and  updating  are  the  responsibility  of  the  materiel 
developer  wtio  has  no  control  over  the  input  from  the  operational 
tester.  Although  there  are  detailed  requirements  for  each  involved 
agency  to  coordinate  with  all  others,  there  is  no  mechanism  for 
deciding  which  critical  issues  will  be  addressed,  or  which  tester 
(DT  or  OT)  will  conduct  what  test.  There  is  a  definite  requirement 
for  an  "honest  broker"  to  insure  that  critical  test  resources  are 


most  effectively  utilized. 


CTP  NOT  COOWPfWRTEO 


•  REGULATIONS 
GOOD 


•  MATERIEL 

OEVELOPER 

RESPONSIBLE 


•NO  DIRECTED 
COORDINATION 


OC5ROR  RESPONSIBLE 

FOR  CTP  MANAGEMENT 


27 


FINDING  AND  RECCM1ENDATI0N 


The  division  of  DA  staff  test  responsibility 
is  inefficient.  DCSRDA  should  be  responsible 
for  OT  and  DT  while  DCSOPS  should  represent 
FDTE. 

Through  the  years,  OCRD  (now  DCSRA)  has  been  the  proponent 
for  Developmental  Testing  on  the  DA  staff.  ACSFOR  had  a  similar 
responsibility  for  User  Testing  (OT  and  FDTE)  until  the  recent 
reorganization  of  the  Army  staff. 

In  reality,  ACSFOR  has  passed  most  of  its  User  test  management 
responsibility  to  its  field  operating  agency,  OTEA.  OTEA  is  primarily 
involved  in  OT  execution,  and  not  located  within  the  Pentagon;  there¬ 
fore,  the  staff  coordination  role  is  inappropriately  assigned. 

Testing  responsibility  on  the  DA  staff  is  now  divided  along 
lines  of  Developmental  Testing  and  User  Testing  rather  than  in  accord¬ 
ance  with  the  division  of  Materiel  Acquisition  Testing  and  Force 
Development  Testing. 

A  spirit  of  confrontation  appears  to  exist  between  the  develop¬ 
mental  and  operational  testers;  therefore,  disputes  over  division  of 
functions  are  difficult  to  resolve. 

DCSRDA,  now  responsible  for  both  R&D  and  acquisition,  should 
assume  responsibility  for  both  DT  and  OT,  which  are  parts  of  the 
acquisition  process,  thus  facilitating  coordination  and  centralizing 
authority.  DCSOPS,  since  it  is  now  responsible  for  force  develop¬ 
ment,  should  assume  staff  responsibility  for  force  tests. 

Both  DCSOPS  and  DCSRDA  should  chair  test  schedule  and  review 
committees  for  their  types  of  tests. 
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FINDING  AND  RECOMMENDATION 


Operational  Teat  responsibility  is  currently 
divided  between  OTEA  and  TRADOC.  The  TECOM 
Boards  should  be  assigned  to  OTEA  and  TRADOC 
should  be  relieved  of  its  OT  responsibility. 

OTEA  plans,  directs  and  evaluates  all  major  and  selected 
non-major  OT.  TRADOC  conducts  other  non-major  OT. 

By  this  division  of  responsibility,  OTEA  is  able  to  direct 
its  attention  to  the  few  acquisition  systems  of  primary  importance. 
This  management  system,  however,  allows  for  varying  policy  and 
direction.  In  addition,  difficulty  may  arise  if  systems  are 
elevated  in  priority.  This  would,  perhaps,  necessitate  a  test 
agency  change. 

The  TECOM  Boards  currently  conduct  the  service-use  phase,  a 
form  of  operational  testing,  which  encroaches  into  the  OT  field. 

By  assigning  the  Boards  to  OTEA,  the  argument  concerning  limits 
of  the  service  phase  of  DT  and  the  OT  would  be  defused  and  OTEA  would 
have  its  own  capability  to  conduct  other  non-major  OT, 

TRADOC  could  then  direct  its  full  attention  to  Force  Testing, 
an  often-neglected  area  of  appropriate  and  critical  interest  zo  that 
command. 
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FINDING  AND  RECOMMENDATION 


There  is  much  confusion  concerning  Che  amount 
of  troop  involvement  in  development  testing. 

AMC,  upon  losing  its  test  boards,  should  retain 
a  limited  capability  to  do  soldier-materiel 
compatibility  testing  as  part  of  DT.  To  avoid 
confusion  as  to  the  meaning  of  this  term,  the 
definition  contained  in  Chapter  VII  should  be 
adopted. 

The  service  phase  of  DT,  as  presently  conducted  by  TECOM, 
is  in  large  measure  duplicative  in  nature  to  OT,  Although  TECOM 
views  the  service  phase  as  answering  questions  about  the  materiel 
while  OT  is  to  answer  questions  which  are  raised  because  new  materiel 
is  being  introduced,  this  definition  does  not  agree  with  the  DOD 
guidance.  In  fact,  the  DOD  requirement  to  accomplish  OT  with 
representative  •  x  troops  in  a  realistic  environment  is  similar 
to  the  specified  criteria  for  the  service  phase.  At  the  same  time, 
there  is  a  definite  requirement  for  the  developer  to  have  some 
user  troops  assist  him  during  the  development  of  the  materiel 
particularly  in  trouble-shooting  the  equipment. 

Many  of  the  answers  required  by  the  developer,  as  his  materiel 
proceeds  through  the  evolution  of  development  and  testing,  can  only 
be  provided  in  an  environment  more  rigidly  controlled  than  that  of 
an  operational  test.  For  this  reason,  AMC  must  have  a  limited  number 
of  user  troops  available,  giving  them  a  capability  to  conduct  soldier- 
materiel  compatibility  testing  as  part  of  normal  development  testing. 
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FINDING  AND  RECOMMENDATION 


Force  Development  Testing  is  currently  frag¬ 
mented.  A  Force  Test  and  Evaluation  Agency 
should  be  foned  under  TRADOC. 

Force  Developaent  Testing  is  currently  conducted  by  two 
potentially  duplicative  agencies,  CDEC  and  MASSTER.  Neither  of 
these  organizations  is  optimally  contrclled  or  directed  by  higher 
command. 

MASSTER  conducts  an  active  test  program,  however,  it  has 
occasionally  been  criticized  for  conducting  tests  not  in  consonance 
with  Army  needs.  It  reports  to  FORSCOM  to  facilitate  troop  support, 
yet  receives  all  its  tasking  from  other  sources — a  violation  of 
proven  managerial  procedure. 

CDEC,  a  highly  instrumented  field  laboratory,  requires  extensive 
time  for  test  planning  and  execution.  This  has  resulted  in  limited 
output.  It  considers  itself  a  totally  impartial  test  agency  which 
does  not  initiate  or  actively  solicit  tests. 

OTEA  has  been  given  responsibility  for  force  test  management; 
however,  its  extensive  involvement  in  the  critical  operational  tests 
has  precluded  active  management  of  force  testing. 

Since  TRADOC  would  be  the  principal  user  of  force  test  data, 
it  would  appear  appropriate  for  that  headquarters  to  command  both 
CDEC  and  MASSTER. 

In  order  to  provide  direction  to  and  coordinate  the  FT  activity 
within  TRADOC,  a  Force  Test  and  Evaluation  Agency  (FTEA)  should  be 
formed.  The  agency  would  task  and  command  both  CDEC  and  MASSTER. 
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SUMMARY  OF  RECOMMENDATIONS 


Implementation  of  study  recommendations 
will  provide  an  improved  system  for  Army 
Test  and  Evaluation. 


•  Give  DCSRDA  staff  responsibility  for  all  acquisition 
testing  (DT  and  OT). 

•  Strengthen  the  CTP  by  giving  DCSRDA  responsibility 
for  coordination. 

•  Consolidate  responsibility  for  all  OT  at  OTEA. 

•  Assign  the  TECOM  boards  to  OTEA  to  assist  in  the 
conduct  of  OT. 

•  Have  TECOM  continue  a  limited  form  of  the  service- 
use  phase  of  DT. 

•  Give  DCSOPS  staff  responsibility  for  all  Force 
Testing. 

•  Give  TRADOC  field  responsibility  for  all  Force 
Testing  and  place  more  emphasis  in  this  area. 

•  Form  from  MASSTER  and  CDEC  resources  a  Force  Test 
and  Evaluation  Agency  under  TRADOC. 

•  Assign  both  CDEC  and  MASSTER  to  the  Force  Test  and 
Evaluation  Agency. 

•  Separate  Force  Testing  into  two  types:  one  for 
testing  materiel  concepts,  the  other  for  testing 
structure  concepts. 

•  Clarify  the  difference  between  goals  of  tests  and 
methods  of  conducting  tests. 
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CHAPTER  I 


INTRODUCTION 

"...  from  tea ting-knowledge  .  .  .  from  knowledge — the  power 
of  decision  ..." 


Goal 

The  goal  of  this  study  is  to  appraise  the  Army's  test  and 
evaluation  organizations  and  procedures  as  they  pertain  to  the 
combat  forces  development  process.  Three  major  objectives 
are  considered  to  achieve  this  goal: 

— Determine  problems  with  the  current  test  and  evaluation 
system  and  organization  for  testing. 

— Develop  a  "best"  generalized  system  for  test  and  evaluation. 

— Determine  the  most  effective  definitions,  procedures,  and 
assignment  of  responsibilities. 

Why  Study? 

Testing  is  big  business  in  the  Army.  It  has  been  estimated  that 
some  $300  million  dollars  and  roughly  18,000  people  are  directly 
chargeable  annually  to  the  test  and  evaluation  sector— excluding 
troop  support. 

At  the  same  time,  as  will  be  shown  in  Chapter  II,  the  subject 
has  received  considerable  high-level  attention,  particularly  over 
the  past  few  years.  As  a  result  of  this  attention,  a  number  of  sig¬ 
nificant  changes  have  been  instituted  which  have  kept  the  system  in 


turmoil.  One  school  of  thought  suggests  «  moratorium  on  further 
change  until  the  present  system  achieves  a  steady-state  before 
additional  study  is  made.  The  test  and  evaluation  system  of  the 
Army,  however,  is  merely  a  small  part  of  the  whole  and  there  have 
been  recent  broad  changes  to  the  whole  system  that  affect — or  will 
affect— the  testing  system  dramatically.  Discussed  in  detail  else¬ 
where,  these  changes  include  the  creation  of  two  major  commands. 

Forces  Command  (FORSCOM)  and  Training  and  Doctrine  Command  (TRADOC) , 
the  reorganization  of  the  Department  of  Army  staff,  and  a  maturing 
Operational  Test  and  Evaluation  Agency  (OTEA)  to  mention  a  few. 

For  these  reasons  alone,  it  is  timely  to  reappraise  the  present 
system  to  determine  if  additional  changes  are  warranted. 

Why  Test? 

Almost  since  the  beginning  of  military  organizations,  tests  of 
some  kind  have  been  run.  It  is  said  that  Xerxes,  the  mighty  King 
of  Persia,  once  built  a  structure  large  enough  to  hold  10,000  men 
wherein  he  "tested"  to  be  sure  his  massive  levys  were  met,  thus 
obviating  the  need  for  a  soldier-by-soldier  count.  A  fundamental 
step  taken  in  the  development  of  organizations,  tactics,  and  doctrine, 
is  a  "test."  War  games,  field  trails,  and  hardware  proofing  have 
long  been  part  of  military  procedure. 

Today  the  Army  tests  for  two  basic  purposes:  First,  as  an  integral 
part  of  the  material  acquisition  processes,  and,  second,  as  a  critical 
means  of  assessing  innovative  ideas  in  organizations,  tactics,  and  doctrine. 
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The  first  category  includes  tests  of  material  under  acquisition 
consideration.  Loosely  grouped  under  terms  such  as  "Developmental" 
and  "User,”  they  Include  tests  to  check  technical  capabilities  of 
in  item,  the  meeting  of  technical  specifications,  logistical  impact, 
hardware  utility  in  a  tactical  context,  quality  assurance,  and  a 
host  of  others. 

Under  the  second  category  are  Included  such  tests  as  those 
conducted  either  in  response  to  the  introduction  of  new  hardware  or 
those  resulting  from  the  formulation  of  new  tactical  and  organizational 
concepts.  In  short,  one  tests  to  gain  knowledge  about  hardware  and 
about  ideas. 


What  is  a  Test? 

Having  briefly  discussed  why  the  Army  tests,  exactly  what  is  a 
test?  Webster  says: 

Test:  Means  of  trial;  to  subject  the  conditions 
that  show  the  real  character  of  thing  in  a  certain 
particular. 

A  test  forms  a  basis  for  evaluation.  An  evaluation  determines  or 
fixes  the  value  of  a  thing  or  an  idea  through  an  examination  and 
judgment  of  test  results.  Testing  is  one  of  the  more  compli¬ 
cated  tasks  in  the  cycle  of  military  research.  Testing  in  the 
broadest  sense  Includes  experimentation  but,  in  many  Instances, 

Army  testing  is  not  scientific  experimentation  in  the  classic  sense; 
it  is,  rather,  an  art.  It  is  an  art  where  scientific  methods  are 
employed  whenever  appropriate,  where  all  judgments  are  supported 
by  as  wide  a  range  of  data  as  possible,  but  where  the  art  will 
usually  be  more  evident  than  the  science. 
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The  Amy  Dictionary,  Ar  310-25,  answers  the  question  "What  is  a 
test?"  as  follows: 

A  test  is  'a  process  by  which  data  are  accumulated 
to  serve  as  a  basis  for  assessing  the  degree  that 
a  system  meets,  exceeds,  or  falls  to  meet  the  tech¬ 
nical  or  operational  properties  ascribed  to  the 
system. ' 

Development  testing  focuses  on  testing  'those 
characteristics  of  equipment  which  pertain  pri¬ 
marily  to  the  engineering  principles  involved 
in  producing  equipment  prosessing  desired  mili¬ 
tary  characteristics  .  .  . * 

Operational  testing  focuses  on  testing  'the 
specific  military  qualities  of  performance 
and  capability  required  of  an  Item  of  equipment 
to  enable  it  to  meet  an  agreed  operational 
need. ' 


^^a^tudy  Group  Approach 

The  three-man  group  undertook  research  that  followed  three  avenues: 

— Reviewed  all  major  previous  studies  and  all  appropriate  regu¬ 
lations. 

—visited  major  activities  and  conducted  face-to-face  discussions 
with  key  personnel  in  the  Army  and  Department  of  Defense  (DOD) . 

Visits  were  made  to  the  Test  and  Evaluation  Command  (TECOM)  at 
Aberdeen  Proving  Grounds;  the  Modem  Amy  Selected  Systems  Test, 
Evaluation  and  Review  (MASSTER)  Headquarters,  Fort  Hood,  Texas;  and 
the  Combined  Arms  Combat  Development  Activity  (CACDA),  Fort  Leaven¬ 
worth,  Kansas.  Key  discussions  were  held  with  LTG  Starbird,  Deputy 
Director,  Defense  Research  and  Engineering  (DDRE) ;  LTG  Deane,  Chief 
of  Research  and  Development  (CRD);  GEN  Depuy,  (TRADOC) ;  MG  McChryalal, 
(MASSTER);  MG  Ochs,  (TEA);  and  MG  Brown,  (TECOM). 
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— Reviewed — on  a  Halted  beals — soae  18  test  reports.  These 
represented  a  saaple  output  froa  OTEA,  HASSTER,  TECOM,  aid  the  Coabat 
Developaents  Experlmentat ion  Coaaand  (CDEC) . 

The  group  also  drew  heavily  froa  their  own  cumulative  17  years' 
experience  in  the  test  and  evaluation  field. 

Scope 

The  study  is  Halted  to  the  Aray.  Further*  it  excludes  the 
testing  activities  of  such  special  organization  as  the  Army  Security 
Agency*  the  Surgeon  General,  Chief  of  Engineers*  and  others  outside 
the  mains treaa. 

The  contents  should  be  viewed  as  a  general  portrait  of  the  system 
rather  than  a  detailed  analysis.  Elements  of  the  system — especially 
organizational  relationships — are  so  dynamic  that  some  will  undoubtedly 
have  changed  prior  to  completion  of  this  effort.  Nevertheless,  the 
underlying  problems  Identified  will  undoubtedly  persist. 


Report  Organization 

In  order  to  provide  a  proper  framework,  it  is  important  to  know 
how  the  Army  arrived  at  today's  testing  system.  Chapter  II  treats 
this  by  discussing  the  evolution  of  testing  in  the  Army  from  a 
historical  perspective,  drawing  heavily  from  the  many  previous  studies 


conducted  in  the  area. 


Chapter  III  sets  forth  the  current  policy  in  terms  of  regulations, 
definitions,  responsibilities,  and  procedures.  The  actual  resources 
dedicated  to  and  available  for  testing  are  the  subject  of  Chapter  IV. 
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The  next  chapter  provides  indications  of  the  future  requirements  for 
testing,  for  it  is  to  meet  these  kncvn  requirements  that  the  Army 
must  be  efficiently  and  logically  organized.  Two  very  recent  events 
impact  upon  this  research  effort  to  a  considerable  degree.  Chapter  VI 
deals  with  these  events,  the  major  reorganization  of  the  Army  staff 
and  the  recommendations  of  a  recently  completed  study  made  for  the 
fecretary  of  the  Army.  Chapter  VII,  entitled  Evaluation,  is  the 
heart  of  the  report,  wherein  the  analysis  of  the  present  system  is 
made  and  the  study  findings  and  recommendations  are  contained. 
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CHAPTER  II 


THE  EVOLUTION  OF  AMff  TEST  AND  EVALUATION 
Background 

The  Any  throughout  its  history  has  been  involved  in  the  testing 
of  new  materiel  and  ideas;  however,  interest  in  the  field  of  test 
and  evaluation  has  Increased  markedly  in  thypast  quarter  century. 

A  change  in  the  Army's  attitude  from  one  of  almost  total  rejection 
of  any  new  idea  to  one  of  active  encouragement  of  research  and 
development  has  had  a  substantial  effect  on  the  test  and  evaluation 
process . 

Although  limited  in  scope,  we  find  an  excellent  example  of  a 
material  acquisition  test  being  conducted  over  one  hundred  fifty 
years  ago.  A  gentleman  by  the  name  of  John  Hall,  conceived  the  idea 
of  a  breech- loading  rifle.  He  developed  and  demonstrated  working 
models  of  the  weapon  and  entered  into  a  contract  with  the  Army  for 
construction  of  several  test  prototypes.  In  1820,  Hall  delivered 
these  weapons  to  the  Army  for  testing.  As  it  turned  out,  tests  indi¬ 
cated  that  the  weapon  was  not  satisfactory  for  use  in  combat  andd, 
therefore,  no  procurement  was  Initiated. 

To  further  illustrate  testing  within  the  Army,  let  us  advance 
the  date  to  1911  when  it  was  concluded  that  the  Army  should  undertake 
some  "experimentation"  which  would  help  develop  a  motor  truck  "to 
replace  the  field  wagon  to  the  greatest  possible  extent."  The 
quartermaster  department  at  that  time  began  a  developmental  and 
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testing  program  to  produce  e  suitable  truck.  A  test  of  two  Army 
trucks  end  e  privately-owned  truck  which  wes  run  from  Washington.  DC 
to  Ft  Benjamin  Harrison  covered  1,524  miles  in  forty-eight  days.  The 
conclusion  was  reached  that  although  the  results  were  not  entirely 
satisfactory,  trucks  did  show  some  promise. 

Lieutenant  General  Lesley  J.  McNair  directed  in  1937  one  of  the 
nos.,  extensive  organizational  tests  ever  conducted  by  the  Army.  As 
a  result  of  these  evaluations,  the  Triangular  Infantry  Division  used 
throughout  World  War  II  was  devised.  Beginning  with  the  fundamental 
study  of  an  Infantry  rifle  squad,  an  entire  divisional  organization 
was  put  together  piece  by  piece.  Matters  on  which  alternative  ideas 
were  tested  Included:  frontages  and  fire  power  per 
man  and  per  unit;  aamunition  allowances;  motor  columns;  the  require¬ 
ment  for  artillery  in  proportion  to  infantry;  the  time  elapsed  in 
transmission  of  orders;  and  the  amount  of  service  support  to  be 
incorporated  in  the  division.  Findings  on  these  organizational, 
tactical  and  doctrinal  questions  were  embodied  in  an  extensive  report 
submitted  by  General  McNair  in  1938. 

These  tests,  along  with  those  of  Colonel  Billy  Mitchell  ir.  1921 
demonstrating  the  effectiveness  of  airpower  against  ships,  provide 
illustrations  of  the  type  of  tests  and  rather  primitive  procedures 
which  existed  prior  to  World  War  II.  Although  testing  was  conducted 
by  the  Army  as  these  examples  indicate,  it  was  not  until  the  late 
1950's  that  the  Army  developed  a  sophisticated  organization  for  test 
and  evaluation. 
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The  remainder  of  this  chapter  trill  deal  with  the  most  recent  15 
years  of  testing  and  will  review  the  evolution  of  testing  using  as 
a  medium  the  various  studies  of  this  subject  made  during  that  period. 

Previous  Studies 

In  the  discussion  which  follows,  we  will  review  only  those  major 
studies  of  the  test  and  evaluation  system  conducted  since  1960.  It 
would,  perhaps  be  helpful  to  review  the  Army's  organization  for  testing 
at  the  beginning  of  the  period.  With  this  preliminary  information, 
one  can  better  evaluate  the  recommendations  of  the  various  study 
groups  and  visualize  the  development  of  the  current  system. 

Test  Organization  in  1960 

A  Department  of  the  Army  publication  entitled  "US  Army  Organiza¬ 
tion  and  Procedures  for  Research  and  Development,"  dated  September  1960 
stated  that  basically  there  were  three  categories  of  tests:  engineering 
tests,  service  tests,  and  troop  tests.  The  publication  also  outlined 
Army  organization  for  conduct  of  these  tests. 

At  that  time  there  existed  the  seven  separate  technical  services — 
their  chiefs  being  heads  of  both  the  Army  technical  staff  and  the 
technical  services.  The  Technical  Service  Chiefs  included  the  Chief 
Chemical  Officer,  the  Chief  of  Engineers,  the  Chief  of  Engineers.,  the 
Chief  of  Ordnance,  the  Quartermaster  General,  the  Chief  Signal  Officer, 
the  Surgeon  General,  and  the  Chief  of  Transportation.  It  was  the 
responsibility  of  each  Chief  of  service  to  develop,  provide,  and 
service  the  material  items  under  his  purview.  His  program  of  material 
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development  was  to  include  engineering  tests  for  new  items  of  equip¬ 
ment.  These  tests  evaluated  the  engineering  characteristics  to  see 
that  the  technical  specifications  were  net. 

The  Continental  Any  Conmander  (CONARC)  had  both  a  Deputy  Chief 
of  Staff  0>CS)  f°r  Materiel  Developaents  and  a  DCS  for  Conbat  Develop¬ 
ments.  The  DCS  for  Materiel  Developments  was  responsible  for  coor¬ 
dinating  and  evaluating  service  tests  conducted  by  the  seven  CONARC 
boards  which  came  under  his  supervision.  The  service  test  evaluated 
an  item  of  materiel  under  simulated  or  actual  operational  conditions 
to  determine  the  degree  to  which  the  item  met  the  stipulated  military 
characteristics  and  its  suitability  for  military  use.  The  CONARC 
Test  Boards  included  the  Artillery,  Armor,  Infantry,  Air  Defense, 

Airborne  and  Electronics,  Aviation,  and  Arctic  Test  Boards.  Although 
primarily  concerned  with  service  testing,  these  boards  also  participated 
to  some  degree  in  troop  tests. 

The  DCS  for  Combat  Developments  had  responsibility  for  combat 
development  field  experiments  and  troop  tests.  He  also  supervised 
the  Combat  Developments  Experimentation  Center,  a  new  field  laboratory 
located  at  Ft  Ord,  California,  which  was  designed  to  prepare,  conduct 
and  evaluate  with  maximum  objectivity  and  scientific  control,  tests 
and  experiments  of  concepts,  organisations,  doctrine,  and  procedures 
for  future  combat.  An  experiment,  although  not  a  specific  category 
of  test,  was  designed  to  be  highly  scientific.  The  troop  test  was 
conducted  in  the  field  by  units  for  the  purpose  of  evaluating  operational 
or  organisational  concepts,  doctrine,  techniques,  procedures,  or  to 
gain  further  information  on  materiel. 
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The  Chief  of  Research  and  Development  had  primary  general  staff 
responsibility  for  the  Army's  engineering  and  service  test  program, 
while  the  Deputy  Chief  of  Staff  for  Operations  (DCSOPS)  had  the  same 
responsibility  for  the  troop  test  and  experimentation  program. 

Project  80  (1961) 

The  Project  80  study  addressed  testing  as  one  aspect  of  the 
materiel  acquisition  and  combat  developments  system.  It  cited  the 
need  for  better  coordination  of  Army  testing  and  recommended  that 
developmental  and  service  testing  be  consolidated  into  one  test 
agency.  As  has  been  pointed  out,  engineering  tests  were  at  that  time 
conducted  by  the  technical  services  while  service  tests  were  carried 
out  by  the  CONARC  boards.  This  recommended  test  agency  would  be 
independent  of  the  developer  and  at  the  same  time  improve  efficiency 
by  consolidating  the  engineering  and  service  tests. 

The  study  also  recommended  the  establishment  of  a  combat  develop¬ 
ments  agency  to  focus  on  the  future.  CDEC  would  be  assigned  to  this 
agency  as  a  concept  test  activity. 

Project  80  stated  that  the  combat  developments  activity  of  the 
DA  staff  (a  function  including  "research,  development,  and  early 
integration  into  the  Army  of  new  doctrine,  new  organization  and  new 
materiel  to  obtain  the  greatest  combat  effectiveness")  should  be 
separated  from  both  the  DCSOFS  and  the  OCRD.  This  proposal  was  the 
genesis  of  a  new  Army  staff  agency  for  force  development. 
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Reorganization  of  the  Army  (1962) 


The  1962  reorganisation  of  the  Any  incorporated  many  of  the 
reco—a ndatlona  of  Project  80.  The  Icey  features  included  the  estab¬ 
lishment  of  an  Army  Material  Command  (AMC)  with  its  own  test  and 
evaluation  command  and  formation  of  the  Coabat ^Developments  Command 
(CDC)  where  future  concepts  wou  ,  be  developed  under  a  single  authority. 

The  Test  and  Evaluation  Coanand  provided  independent  engineering 
and  service  tests  for  the  commodity  commands  (the  new  name  for  the 
technical  services  now  subordinate  to  AMC).  This  centralized  test 
command  assumed  control  of  the  CONAKC  test  boards  and  all  test 
agencies  of  the  technical  services.  It  was  designed  to  reduce 
requirements  for  lateral  coordination  and  to  shorten  lead  time.  TECOM 
also  assumed  responsibility  for  supervising  the  troop  tests  conducted 
by  CON  ARC. 

The  Combat  Developments  Experimentation  Center  was  assigned  to 
CDC  and  given  the  mission  of  concept  testing  and  field  experimentation. 

In  summary,  test  responsibility  under  this  new  organization  was 
as  follows:  AMC  -  engineering  and  service  testing;  CONARC  -  troop 
testing  (supervised  by  AMC/TECOM);  and  CDC  -  concept  tests  and  field 
experiments. 


Sate  (1966) 

The  study  of  Army  Test  and  Evaluation  (SATE)  was  Initiated  by  a 
memorandum  from  the  Army  Chief  of  Staff  in  October  1965.  The  objective 
was  to  determine  how  the  Army  could  best  organize,  administer  and 
control  the  test  and  evaluation  process  to  support  the  timely  acquisition 
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of  equipment.  Interestingly  enough,  the  study  criticized  the  1962 
ssslgnnent  of  engineering  and  service  testing  to  AMC/TECOM,  stating 
that  the  "developer,"  AMC,  vss  now  responsible  for  the  total  evaluation 
of  its  own  product.  It  is  noteworthy  that  the  1962  reorganization 
had  looked  upon  the  technical  services  as  the  "developers"  and  had 
atteaptcd  to  eliminate  the  technical  services  evaluation  of  their 
own  products  by  the  formation  of  TECOM  as  an  independent  test  agency. 

SATE  cited  the  need  for  improved  operational  testing  within  the 
Army  and  recommended  the  assignment  of  TECOM  to  CDC  where  all  material 
testing  would  be  conducted.  This  was  an  attempt  to  separate  operational 
testing  from  the  material  developer  and  provide  the  decisionmaker 
with  an  Independent  user  evaluation.  It  failed,  however,  to  recognize 
the  developer’s  need  to  accomplish  engineering  tests.  The  report 
also  pointed  out  the  need  to  minimise  the  number  of  activities  and 
installations  involved  in  testing  and  the  requirement  to  simplify 
the  cumbersome  regulations  governing  testing. 

Among  the  other  "improvement  actions"  were  recommendations  for 
advanced  instrumentation,  scientific  support  and  statistical  analysis 
in  service  tests;  greater  participation  by  service  schools  in  test 
planning  so  as  to  Include  doctrine  in  the  coobat  development  process; 
and  the  assignment  of  DA  staff  responsibility  for  the  "life  span  of 
materiel"  to  the  Assistant  Chief  of  Staff  for  Force  Development  (ACSFOR) . 

The  Brown  Board  (1967) 

A  "Board  of  Inquiry  on  the  Army  Logistics  System"  was  established 
in  September  of  1965  with  Lieutenant  General  Frederic  J.  Brown  as 
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Chairman.  Its  purpose  we*  to  analyse  the  entire  Aray  logistics  system 
to  determine  what  changes  and  modifications  were  needed.  Only  a  single 
section  of  this  six-volume  report  dealt  with  test  and  evaluation. 

The  principal  thrust  of  the  report  was  that  the  "systems"  approach 
to  testing  was  being  neglected  in  favor  of  a  "hardware"  approach. 

By  this  the  report  meant  that  too  much  emphasis  was  being  given  to 
the  functioning  of  the  materiel  and  not  enough  to  its  relationship 
with  the  enemy  threat,  future  doctrine,  environment,  organization, 
training,  and  logistic  support  requirements.  In  short,  the  report 
seems  to  indict  the  testers  for  their  isolation  from  the  field  army 
and  their  narrow  view  of  testing. 

The  Board  pointed  out  that  testing  had  been  accorded  preeminence 
over  the  function  of  evaluation.  In  reality  it  said  that  "evaluation" 
had  become  little  more  than  "test  assessment"  and  that  a  broader  view 
of  evaluation  was  required. 

Significant  findings  were: 

— Controls  over  test  programs  and  test  schedules  are  generally 
inadequate . 

— Test  reports  are  not  required  for  DA  decisions,  but  thorough 
evaluations  are  essential. 

—There  is  no  comprehensive  test  program  which  embraces  all 
phases  of  acquisition  management. 

In  summary,  the  Brown  Board  recommended  substantial  changes  in 
test  philosophy  and  dealt  lightly  with  organizational  relationships. 
The  Board  believed  that  testing  had  two  primary  roles:  (1)  Provide 
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data  to  guide  develop— nt  and  (2)  provide  data,  the  assess— nt  of 
which  Is  one  Input  to  evaluation.  It  is  worthy  of  note  that  — ny  of 
the  philosophical  flaws  found  In  tha  test  system  by  the  Brown  Board 
continue  to  exist. 


The  Fulton  Cn— Ittce  (1969) 

Under  the  leadership  of  BG  Will lea  Fulton,  a  study  was  initiated 
to  review  the  integration  of  the  Defense  Co—unications  Planning 
Groups  responsibilities  into  the  Army.  The  committee  also  focused 
on  the  lack  of  a  responsive  operational  test  and  evaluation  capability 
within  the  Army. 

In  particular,  the  group  sought  to  expedite  the  review  and  testing 
of  surveillance,  target  acquisition  and  night  observation  (STANO) 
equip— nt  which  was  assentlal  to  activities  in  Vietnam.  As  a  result 
of  the  study.  Project  MASSTER  (Mobile  Army  Sensor  Systems  Test  Evalua¬ 
tion  and  Review)  was  established  at  Ft  Hood,  Texas,  with  a  mission 
of  providing  centralised  control  of  all  test  activities  dealing  with 
STANO  ite— .  This  manage— nt  technique  reduced  reaction  time s  and 
facilitated  coordination  among  Army  commands.  MASSTER  was  assigned 
to  CONARC  in  order  to  facilitate  troop  support  of  testing. 

In  1971,  Project  MASSTER  expanded  its  function  from  that  of  only 
testing  STANO  devices  to  one  of  general  force  development  testing  and 
as  a  result,  changed  its  na—  to  Modem  Army  Selected  Syste—  Test 
Evaluation  and  Review  (still  MASSTER). 

This  broadening  of  scope  for  MASSTER  gave  the  Army  two  force 
development  test  centers,  often  duplicative  in  functioning  and  equipment. 


It  became  generally  accepted  that  MASSTER  would  perform  large  unit/ 
subjective  tests  while  CDEC  would  conduct  small  unit/objective 
experiments . 


The  Parker  Panel  Report  (1970) 

The  Parker  Panel ,  a  well  publicised  study  of  the  Army  organization 
in  1971,  recommended  combining  the  CONARC  schools  and  the  doctrinal 
agencies  of  CDC  under  the  comand  of  CDC.  By  this  action,  the  Panel 
believed  that  doctrine  and  training  could  be  better  integrated  and 
that  the  functioning  of  both  CDC  and  CONARC  would  be  enhanced.  The 
Panel  also  concluded  that  the  Army  could  not  afford  to  continue  operating 
CDEC  and  MAFSTER  as  separate  agencies  with  a  potential  duplication 
of  capability  and  without  integrated  test  planning.  It  reconmended 
combining  the  two  facilities  under  CDC  in  order  to  facilitate  coor¬ 
dination  and  planning. 

Neither  of  the  recommendations  of  the  Parker  Panel  was  immediately 
Implemented;  however,  an  Army  reorganization  three  years  later,  in 
1973,  did  consolidate  doctrine  and  training  under  a  single  command.  , 

USACDC  CDEC/MASSTER  Study  (1970) 

Shortly  after  the  Parker  Panel  report  was  published  recommending 
the  combining  of  CDEC  and  MASSTER,  the  Commanding  General  of  CDC 
directed  a  study  to  determine  the  optimum  organization  of  this  combined 
element.  This  study  found,  however,  that  it  was  impractical  to  move 
or  disband  either  of  the  units  and  thus  recommended  the  formation  of 
a  test  conti&nd  which  would  include  both.  This  action  would,  according 
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to  the  report,  insure  coordination  of  effort  and  eliminate  the 
fragmentation  of  limited  expertise.  It  also  recommended  that  con¬ 
sideration  be  given  to  assigning  the  test  boards  of  TEGOM  to  CDC. 

Blue  tlbbon  Defense  Panel  (1970) 

Mr.  Gilbert  V.  Fitzhugh  was  appointed  chairman  in  July  1969  of 
a  special  panel  to  study  the  organisation  and  management  of  the  entire 
Department  of  Defense.  In  making  this  appointment.  President  Nixon 
gave  this  prestlgeous  panel  a  broad  charter  which  Included  evaluation 
of  testing  within  the  Defense  establishment. 

The  Panel  reported  that  developmental  testing,  "functional  testing" 
as  they  called  it,  was  being  conducted  well  in  all  Services.  On  the 
other  hand,  they  were  extremely  critical  of  operational  testing  and 
recoomended  a  number  of  actions  to  improve  procedures.  Included 
among  these  were: 

— Establishment  of  a  Defense  Department  Operational  Test  and 
Evaluation  (OTE)  Group  with  civilian  leadership,  reporting  directly 
to  the  Secretary  of  Defense. 

—Establishment  of  a  Defense  Test  Agency  with  broad  authority 
and  responsibility  for  DOD  test  activities. 

—Increased  emphasis  on  OTE. 

— Improved  joint  OTE. 

—Separate  budgeting  by  Services  for  OTE. 

Although  the  Secretary  of  Defense  did  not  establish  the  OTE  Group 
or  the  Defense  Test  Agency  in  accordance  with  the  past  two  recommenda¬ 
tions,  he  did  give  emphasis  to  the  importance  of  OTE.  This  emphasis 
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be case  the  guiding  and  predominant  stress  which  is  felt  by  the  Services 
even  today. 

The  Army  responded  to  the  recommendations  of  the  Blue  Ribbon 
Psnel  by  initiating  what  was  called  an  Expanded  Service  Test.  This 
test  eaphasized  the  operational  environment  but  was  conducted  by 
TECOH,  for  all  practical  purposes  the  developer.  This  modification, 
of  course,  did  not  satisfy  the  Department  of  Defense  which  would 
strive  in  the  next  few  years  for  increased  independence  of  operational 
testing. 


Priority  Project  Four  (1972) 

Before  reviewing  other  Army  actions  directed  at  stressing  OTE, 
it  would  be  beneficial  to  consider  one  final  study  of  Army  "test, 
evaluation  and  field  experiments."  In  1971,  a  study  group  was  assembled 
to  assist  the  Comptroller  of  the  Army  in  making  recommendations  to 
the  Chief  of  Staff  for  improving  conduct  of  operational  test  and 
evaluation. 

The  report  of  the  group  concluded  that  the  establishment  of  an 
Army  test  command  for  operational  and  field  testing  was  appropriate. 

Also  among  its  recommendations  were: 

— Better  definition  of  test  responsibilities. 

— Consolidation  of  all  test-associated  Army  regulations  into  one 
test  and  evaluation  regulation. 

—Elimination  of  TECOM  headquarters. 

— Closure  of  CDEC  with  gradual  transfer  of  its  mission  to  MASSTER. 

—  Improved  test  forecasting  to  enhance  effective  management. 

ss< 
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Th«  Army  Baorganisatlon  (1973) 


Just  prior  to  tbs  major  rsorganlsstlon  of  the  Army,  and  in  accord¬ 
ance  with  the  Blue  Ribbon  Defense  Panel  study,  DOD  directives  end 
Priority  Project  four  reconaendations,  the  Army  established  an 
Operational  Test  and  Evaluation  Agency  (OTEA)  under  staff  supervision 
of  ACSFQR,  but  reporting  its  results  directly  to  the  Chief  of  Staff. 

This  agency  provided  a  strong  focal  point  for  operational  testing 
within  the  Army  end  planned,  directed  and  evaluated  the  operational 
tests  of  certain  important  systems.  By  the  formation  of  this  agency, 
the  Army  had  complied  with  the  DOD  guidance  to  conduct  operational 
tests  in  an  organisation  separate  and  distinct  from  the  developing/ 
procuring  command. 

The  reorganisation  of  the  Army  in  1973  formed  the  Training  and 
Doctrine  Command  and  the  Forces  Conmand  from  the  old  Continental  Army 
Command  and  the  Combat  Developments  Command.  Thus  the  proposed 
combining  of  doctrine  and  training  into  one  command  as  the  Parker 
Panel  had  recommended,  finally  became  a  reality. 

Chapter  IV  will  discuss  in  detail  the  current  organization  for 
teat  and  evaluation.  Suffice  it  to  say  here  that  many  of  the  recommenda¬ 
tions  of  studies  conducted  in  the  past  were  Incorporated  in  the  Army 
reorganisation  of  1973. 


se< 
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CHAPTER  III 


CURMHT  A MR  TEST  AMD  EVALOATIOW  POLICIES 

General 

This  chapter  discusses  the  Amy's  current  test  and  evaluation 
system,  to  include  regulations  under  which  test  and  evaluation  is 
conducted*  and  the  specific  nature  and  type  of  tests  to  I/e  conducted 
under  this  system. 


POD  Direction 

During  1970  and  the  first  eight  months  of  1971,  Deputy  Secretary 
of  Defense  Packard  made  sweeping  changes  in  the  Services  materiel 
acquisition  procedures.  On  one  hand,  he  delegated  to  the  Services 
more  responsibility,  but  on  the  other  hand,  he  directed  that  more 
attention  be  given  to  performance  testing.  Mr.  Packard's  guidance 
which  was,  in  part,  based  on  the  recommendations  of  the  Blue  Ribbon 
Defense  Panel,  included  the  establishment  of  the  Office  of  Deputy 
Director  for  Test  and  Evaluation  in  the  Office  of  Defense  Research 
and  Engineering.  The  Deputy  Director  was  given  overall  responsibility 
for  test  and  evaluation  policies  and  procedures  for  DOD.  In  January 
1973,  DOD  Directive  5000.3,  Test  and  Evaluation,  was  published  as 
guidance  to  the  Services.  This  directive  provided  that: 

— Test  and  Evaluation  shall  be  commenced  as  early  as  possible  and 
conducted  throughout  the  system  acquisition  process  as  necessary  to 
assist  in  progressively  reducing  acquisition  risks  and  in  assessing 
military  worth. 
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— Acquisition  schedule*  will  be  based  upon  eccoapllshing  test 
and  evaluation  Milestones  prior  to  the  tine  that  key  decisions 
regerdlng  procurement  are  made. 

— Before  the  initiation  of  developnent  of  a  new  system,  test  and 
evaluation  using  existing  systems,  or  modifications  thereto,  nay  be 
appropriate  to  help  define  the  military  need  for  the  proposed  new 
system  and  to  estimate  its  military  worth. 

The  Directive  established  the  principal  types  of  tests  to  be 
conducted  as  Development  Test  and  Evaluation  (DTE),  Operational  Test 
and  Evaluation  (OTE),  and  Production  Acceptance  Test  and  Evaluation 
(PATE).  These  tests  were  defined  as  follows: 

—DTE  is  that  test  and  evaluation  conducted  to  demonstrate  that 
the  engineering  design  and  development  process  is  complete;  that  the 
design  risks  have  been  minimized;  that  the  system  will  meet  specifica¬ 
tions;  and  estimate  the  system's  military  utility  when  introduced. 

— OTE  is  that  test  and  evaluation  conducted  to  estimate  the  pro¬ 
spective  system's  military  utility,  operational  effectiveness,  and 
operational  suitability  (including  compatibility,  interoperability, 
reliability,  maintainability,  and  logistic  and  training  requirements), 
and  need  for  any  modifications.  In  addition,  OTE  provides  Information 
on  organisation,  personnel  requirements  doctrine  and  tactics. 

— PATE  is  that  test  and  evaluation  of  production  i terns  tc  demonstrate 
that  the  items  procured  fulfill  the  requirements  and  specifications 
of  the  procuring  contract  or  agreements. 


Essentially*  0IE  procedures  were  not  effected  by  the  directive; 
however*  the  Services  were  given  specific  gulden ce  on  the  conduct 
of  OTE.  Of  nejor  iaportsnce  were  the  following: 

— Each  Service  will  have  one  nejor  field  agency  separate  end 
distinct  from  the  developing  cows end  which  will  report  its  test  snd 
evaluation  results  directly  to  the  Service  Chief*  recover nd  the 
accoepllshaent  of  adequate  OTE  snd  insure  that  OTE  is  effectively 
planned  and  conducted. 

— Each  Service  will  have  a  full-tine*  strong*  focal  point  on  its 
headquarters  staff  to  assist  the  independent  OTE  field  agency. 

—As  a  ninlnun,an  initial  phase  of  OTE  will  be  conducted 
prior  to  the  first  nejor  production  decision  in  order  to  provide  a 
valid  estlnste  of  systen  operational  effectiveness  end  suitability. 

For  complex  systems,  additional  phases  of  OTE  may  be  required  prior 
to  the  first  nejor  production  decision. 

— When  sufficient  production  items  become  available*  follow-on 
phases  of  OTE  will  be  accomplished  in  order  to  fully  determine  the 
systems  operational  effectiveness  and  suitability. 

* 

Army  Direction 

To  comply  with  Mr.  Packard's  guidance  on  improving  the  materiel 
acquisition  process,  the  Army  published  AR  1000-1  and  a  Letter  of 
Instruction  (LOI)*  dated  23  August  1972,  promulgated  the  Army's  basic 
policies  for  systems  acquisition  which  sought  to  minimize  costs,  shorten 
development  time,  and  assure  adequate  performance.  These  basic  policies 
were  dealt  with*. 
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—Shortened  requirements  generation  time. 

— High-level  decisionmaking. 

— Shortened  development  time. 

— Funding  priorities. 

—Cost  versus  quantity. 

— Program  cost  control. 

Of  major  importance  to  the  Army  policy  for  test  and  evaluation 
was  the  decision  to  establish  an  Army  Systems  Acquisition  Review 
Council  (ASARC)  paralleling  the  already  established  DOD  Defense 
System  Acquisition  Review 'Cbuncil  (DSARC) .  Figure  one  shows  the  level 
of  decision  for  the  various  types  of  acquisition.  ASARC  meetings 
are  scheduled  to  coincide  with  the  developmental  cycle  of  the  system: 


Milestone 

Meeting 

Enter  Validation 

ASARC  I 

Enter  Full-Scale  Engineering 

ASARC  II 

Development 

Low  Rate  Initial  Production 

ASARC  I I -a 

Full-Scale  Production 

ASARC  III 

Development  testing  (DT)  and  operational  testing  (OT)  are  planned 
and  conducted  so  that  the  results  can  be  provided  to  the  appropriate 
ASARC.  As  an  example,  the  policy  states  that  for  ASARC  11a  the  basic 
output  of  DT  and  OT  should  answer  the  question,  "Is  the  system  basically 
OK,  can  any  identified  deficiences  be  corrected,  and  if  corrected,  do 
we  want  the  system  considering  what  we  already  have?"  The  policy 
also  states  that  the  final  OT  of  low  rate  initial  production  units  is 
Intended  to  determine: 
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LEVELS  ©F  ©EOSTO 
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Fig,  1,  Levels  of  Decision 
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(X)  The  overall  system's  field  effectiveness  in  the  hands  of 
troops  (benefits  vs  burdens)  to  Include  performance  against  expected 
countermeasures . 

(2)  The  system's  maintainability  and  reliability  within  the  limits 
of  the  test  period. 

(3)  The  readiness  of  the  system  for  deployment  in  terms  of  basis 
of  issue,  organization,  tactics,  and  the  training  package. 

In  the  policy  to  shorten  development  time,  testing  was  divided 
into  only  the  two  categories — DT  and  OT.  The  policy  further  stated 
that  OT  would  be  conducted  by  AMC  and  would  include  engineering  testing 
and  only  that  part  of  service  testing  which  assesses  operability  and 
maintainability  of  the  system  by  the  prospective  user.  It  further 
directed  that  OT  would  be  conducted  by  troops  or  Indi.^uuals,  preferably 
in  units,  to  determine  if  the  system  is  operationally  suitable  from 
a  doctrinal,  organizational,  and  tactical  point  of  view.  Figure  two 
graphically  presents  the  entire  system  acquisition  cycle. 

Current  DA  Regulations 

With  the  exception  of  AR  1000-1,  which  was  published  on  13  July  1972, 
the  Army  has  been  very  slow  In  providing  guidance  for  the  new  test 
procedures.  The  basic  regulation  of  Research  and  Development,  AR  70-1, 
was  published  in  March  1973.  The  section  on  testing  was  limited  to 
a  repeat  of  the  guidance  in  AR  1000-1,  with  the  addition  of  DA  staff 
responsibility  for  DT  being  assigned  to  the  CRD  and  OT  to  the  ACSFOR. 

On  15  January  1974,  DA  published  AR  10-4,  which  provided  the  mission, 
functions,  and  command  relationships  of  OTEA  and  established  official 
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SYSTEM  ACQUISITION  CYCLE 
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Fig.  2,  System  Acquisition  Cycle 
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definitions  of  User  Testing,  Operational  Testing,  Force  Development 
Testing  and  Experimentation  (FDTE)  and  Joint  User  Testing. 

— User  testing  was  defined  as  a  generic  term  which  encompassed  OT, 
FDTE,  and  Joint  User  Testing. 

— OT  was  as  previously  defined  In  the  LOI. 

— FDTE  was  defined  as  those  tests,  ranging  from  the  small  in  scope, 
highly  instrumented,  high  resolution  field  experiments  to  the  broader 
in  scope,  less  instrumented,  low  resolution  and  highly  subjective 
field  tests  that  are  performed  to  support  the  force  development  process 
by  examining  the  impact,  potential  and  effectiveness  of  selected  con- 
cepts,  tactics,  doctrine,  organization  and  materiel.  FDTE  may  support 
the  materiel  acquisition  process  by: 

— Providing  data  to  assist  in  the  development  of  a  Required 
Operational  Capability  (ROC). 

— Develop  data  to  fully  understand  the  performance  of  a 
materiel  system. 

—Validate  doctrine  and/or  tactics  to  confront  a  possible 
threat  response  to  a  deployed  system. 

— Joint  User  Testing  is  merely  that  testing  in  which  the  Army 
participates  with  another  Service. 

Annex  H,  Test  and  Evaluation  of  the  LOI  prescribed  policy  and 
provided  guidance  for  the  conduct  of  test  and  evaluation.  This  ann  < 
divides  both  DT  and  OT  into  three  separate  tests— I,  II,  and  III. 

The  policy  in  this  annex  directs  that  DT  should  first  test  components, 
then  subsystems  and  finally  prototypes  or  preproduction  models  of  the 
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entire  system.  It  further  directs  that  DT  will  include  "soldier 
proofing"  through  participation  of  representative  user  personnel. 

On  the  OT  side,  the  policy  directs  that  OT  is  conducted  as  necessary 
and  as  early  as  practicable,  with  early  prototypes  and  continuing 
through  production.  Annex  H  defines  the  three  DTs  and  three  OXs  as 
shown  on  figure  three. 

Regulations  Under  Revision 

As  a  result  of  the  new  policies  for  materiel  acquisition,  the  basic 
regulations  governing  testing,  AR  70-10,  Test  and  Evaluation  During 
Development  and  Acquisition  of  Kateriel  and  AR  71-3,  Joint  User  and 
Force  Development  Tests,  Experiments,  and  Evaluations,  were  rescinded 
and  new  regulations  have  been  under  revision  to  date. 

AR  70-10  -  The  latest  draft  is  dated  7  November  1973  and  contains 
detailed  guidance  fot  the  conduct  of  DT  and  OT.  In  the  draft  AR, 
the  characteristics  of  DT  and  OT  are  consistent  with  those  in  DOD 
Directive  5000.3;  however,  the  draft  regulation  gives  considerable 
emphasis  to  the  fact  that  the  OTE  process  will  be  independent  and 
normally  separate  from  the  DTE  process.  Although  the  draft  AR 
specifies  that  DT  II,  0T  II,  and  DT  III  will  be  conducted  on  all 
development  items/systeos  as  a  minimum,  the  tone  is  such  that  the 
full  series  of  DT  and  0T  would  normally  be  conducted  on  all  development 
items  regardless  of  their  classification  as  major,  selected  nonmajor 
or  other  nonmajor  systems. 

The  DT  portion  of  the  draft  regulation  directs  "that  DT  will  be 
accomplished  under  carefully  controlled  conditions  utilizing  scientific 
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1  ail  test  la  conducted  early  l»  the  dewe  Ispmcnt  cycle,  normally 

during  the  Valldatiou  I1u».  Ca^niwiti,  subsystems,  or  the  entire 
Ijwtn  arc  turinci  to  determine  whether  the  systra  li  ready  for 
rail-scale  Developm-nt.  This  tret  stay,  ia  the  ease  oi  cwapctitl VC 
cyatraa,  provide  a  comparison  he  tween  the  systems  tested.  Uhcra 
appropriate,  aperatiaoal  testing  I*  c  undue  ted  ceacarreatly  with 
this  teat. 


II  this  teat  provides  the  technics!  date  necessary  to  assess  whether 
live  system  is  ready  (nr  product  list.  It  Measures  the  technical 
performance  end  safety  characteristics  of  tits  I  tecs  and  its  associ¬ 
ated  tools,  test  epilpwat,  training  -sekage,  and  amintcnance  test 
packacc  as  described  in  the  DP.  T'  —ileal  reliability  and  main- 
taiaability  will  also  he  assessed  during  this  test.  The  test 
cnconpasses  all  the  cleav.-nts  of  the  fornerly  designated  Engineer¬ 
ing  Test/Eapanded  Service  Test  (ET/EST)  except  for  tfie  field  test 
uith  a  trsap  unit.  OT  II  will  include  "soldier-proof ing"  by 
representative  user  personnel  but  not  necessarily  in  a  truly  oper¬ 
ational  environment.  Operational  testing  Is  norvtal ly  conducted 
concurrent iy  with  DT  II  by  the  designated  ce— and  or  agency  ia 
coordination  uith  the  Materiel  developer's  test  c mound. 


Ill  This  test  is  conducted  on  systeos  froo  Che  initial  production  run 
to  verify  that  the  tysten  Meets  the  specifications  prescribed  for 
it.  The  test  elso  serves  to  confirm  that  deficiencies  found  in 
DT  II  have  been  corrected  and  has  the  sane  scop#  and  purpose  as 
specified  la  At  70-10  for  the  Initial  Production  Teat. 


OPEKATIONAL  TEST? 


I 

» 

This  test  provides  early  Information  as  to  systen  operational  suit¬ 
ability,  sad  s  cooperison  to  existing  systems,  in  order  to  assist  in 
determining  whether  the  system  should  enter  Full-Scale  Development. 

■  - OT  I  may  also  help  identify  or  refine  critical  issues  to  be  examined 
in  subsequent  operational  testing.  In  those  cases  where  the  oppor¬ 
tunity  exists  for  the  conduct  of  OT  I  -  for  example,  where  competi¬ 
tive  prototypes  or  well  advanced  prototypes  exist  -  it  will  be  con¬ 
ducted  concurrently  with  DT  1  using  a  single,  coordinated  test  plan. 

11 

This  test  ia  accoaplifhed  prior  to  the  production  decision  (ASAXC  Ila/ 
DSAXC  lie  for  najor  systems)  end  provides  an  assesssient  of  system 
operational  suitability  and  effectiveness.  It  also  provides  informa¬ 
tion  needed  to  refine  or  validate  organisational  and  employment  con¬ 
cepts  and  determine  training  and  logistic  requirements.-  ’  OT  11  is 
normally  accomplished  concurrently  with  DT  11,  using  complete  pre- 
production  prototypes.-  Complete  interchange  of  information  and  data 
obtained  during  DT  11  and  OT  11  is  mandatory.  During  OT  II,  the  systen 
Is  subjected  to  s  realistic  operational  environment,  using  a  small 
troop  unit  typical  of  that  which  will  ultimately  be  equipped  with  the 
syeten.  OT  II  will  produce  sufficient  and  timely  results  to  allow  for 
an  Independent  evaluation  to  be  available  to  aselst  In  making  •  Low 
gate  Initial  Production  decision  at  ASARC  Ila/DSARC  Its  for  major 
systems ,  or  t  production  recommendation  at  the  IK  for.  other  eys tens. 

lit 

Ihle  test  is  accomplished  using  early  production  models  and  provides 
information  to  refine  or  validate  earlier  estimates  of  operational 
effectiveness,  to  determine  the  operational  suitability  of  the  produc¬ 
tion  model,  to  optimise  organisation  end  doctrine,  to  validate  train¬ 
ing  and  logistic  requirements,  end  to  identify  any  additional  actions 
that  should  be  taken  before  the  mv  system  is  dsptoyed. 

Pig,  3,  Definition  of  DT  and  0T 
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techniques,  instruaentstion,  statistical  methodology,  and  simulation." 
The  regulation  does  provide,  however,  that  typical  user  weapon  and 
vehicle  crews  may  be  used  to  accomplish  human  factors  tests  to 
observe  the  man  (soldier) -machine  interface  and  that  interface  between 
military  operators  and  the  maintainers.  The  regulation  also  divides 
DT  II  into  two  phases  in  which  engineering  and  service-use  aspects 
are  examined.  The  service-use  phase  provides  for  a  technical  estima¬ 
tion  of  military  utility  and  is  performed  under  limited  and  specific 
field  conditions  representative  of  the  anticipated  usage  of  the  equip¬ 
ment.  The  draft  AR  does  direct,  however,  that  the  service  phase  will 
be  coordinated  closely  with  OT  II  to  preclude  duplication  and  insure 
complete  examination  of  all  critical  issues. 

In  the  detailed  guidance  for  OTE,  the  draft  regulation  specifies 
that  OT  will  be  conducted  under  realistic  operational  conditions  using 
tactical  scenarios  and  TOE  troop  units.  At  the  same  time,  the  draft  AR 
directs  that  "conditions  will  be  controlled  utilizing  scientific 
test  techniques,  instrumentation,  statistical  methodology  and  simula¬ 
tion  and  subjective  military  judgment  of  user  personnel  to  assure 
validity  and  permit  analysis  of  results."  In  the  guidance  for  OT  II 
and  OT  III,  it  further  directs  that  the  operationally  critical  issues 
be  examined  by  testing  in  TOE  troop  unit  field  exercises.  The 
idea  of  combining  extensive  scientific  techniques  with  troop  unit  field 
exercises  is  not  compatible  and  is  difficult,  if  not  impossible,  to 
accomplish. 

The  last  major  portion  of  AR  70-10  (draft)  provides  guidance  for 
the  Coordinated  Test  Program  (CTP) .  The  CTP  is  the  principle  management 
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docuaent  for  assuring  that  appropriate  DT  and  OT  are  properly  planned, 
coordinated,  conducted  and  reported.  The  CTP  has  the  making  of  a 
fine  management  tool,  however,  its  preparation,  coordination,  distri¬ 
bution  and  updating  are  the  responsibility  of  the  materiel  developer, 
who  has  no  control  over  the  input  from  the  operational  tester.  The 
command  agency  responsible  for  OT  prepares  the  chapter  of  the  CTP 
which  lists  all  the  critical  issues  to  be  answered  by  OT  and  how  they 
will  be  resolved.  Figure  four  shows  the  content  of  the  various 
sections  of  the  CTP. 

AR  71-3  -  The  other  regulation  in  draft  provides  guidance  for  the 
programming,  planning,  budgeting  and  reporting  of  FDTE  and  Joint  User 
Testing.  The  guidance  contained  in  this  draft  has  been  nullified 
by  the  reorganization  of  the  DA  staff  on  20  May  1974. 

Five  Year  Test  Program 

The  FYTP  is  a  compendium  of  approved  outline  test  plans  for  user 
testing  (OT's  and  FDTE's).  It  is  a  tasking  document  for  execution  for 
the  current  fiscal  year  plus  one  and  a  planning  document  for  the 
remaining  years.  It  is  updated  and  published  semiannually  by  OTEA. 

Each  outline  test  plan  contains  the  test  purpose,  objective,  scope,  and 
personnel/equipment  resource  requirements,  and  tactical  context. 

As  such,  the  FYTP  provides  management  visibility  as  to  exactly  what 
is  being  tested  and  why.  Outline  test  plans  are  processed  and 
challendged  by  a  multi-command/agency  general  officer  representative 
body  called  the  Test  Schedule  Review  Committee  (TSARC) . 

The  TSARC,  chaired  by  the  Commander  of  OTEA,  coordinates  and  schedules 
overall  resource  requirements  for  user  testing.  General  officer  members 


31 


68 


CHAPTER  CONTENTS  PREPARED  PROVIDE  INPUT 


I  DT  PLAN  MATERIEL  MATERIEL 

DEVELOPER  DEVELOPER 


2  OT  PLAN  COMMAND/  COMBAT 

AGENCY  DEVELOPER, 

RESPONSIBLE  TRAINER, 

FOR  OT  USER, 

LOGISTICIAN 


3  NON-RDTE  MATERIEL 

FUNDS  DEVELOPER 

AND  COMMAND/ 

I  I  V  »%  W  I 

KtbHUNb I  bit 

FOR  OT 


4  RDTE  MATERIEL 

FUNDS  DEVELOPER 

AND  COMMAND/ 
AGENCY 
RESPONSIBLE 
FOR  OT 

NOTE:  Each  chapter  to  be  coordinated  by 
materiel  developer,  user,  command/ 
agency  responsible  for  OT,  logistician, 
combat  developer,  trainer  and  test 
support  unit. 


fig.  4,  COORDINATED  TEST  PROGRAM 
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of  the  TSARC  represent  OCRD,  DCSOPS,  DCS LOG,  TRADOC,  FOES COM,  and  AMC. 
MASSTER  end  CDZC  also  provide  appropriate  representatives.  The  committee 
meets  semiannually  In  June  and  December.  A  complete  description  of 
the  TSARC/FYTP  Is  contained  In  AR  10-4  and  AR  71-3  (draft) . 


CHAPTER  IV 


ORGANIZATION  AND  RESOURCES  FOR  TEST  AND  EVALUATION 

Introduction 

Now  that  the  evolution  of  A ray  testing  has  been  discussed  and 
the  policies  which  govern  this  testing  reviewed,  it  is  appropriate 
that  the  current  organization  an/  resources  for  this  activity  be 
studied-in  soae  detail.  As  a  general  statement,  it  would  be  proper 
to  say  that  for  test  and  evaluation,  DOD  provides  the  broad  policy 
direction  while  DA  Insures  the  coordination  and  managenent  of  effort. 

The  actual  testing  is  conducted  by  various  agencies  and  commands 
throughout  the  Army. 

This  overview  of  test  organization  will  discuss  each  level  of 
command  and  separate  agency  involved  in  testing,  outlining  its 
function  and  providing  a  brief  summary  of  the  resources  available. 

The  type  of  testing— developmental,  operational  or  force  development- 
conducted  by  the  organization  will  be  stressed.  Although  not  reflected 
in  the  discussion  of  this  chapter,  ACSFOR  and  OCRD  were  abolished  on 
20  May  1974.  Chapter  VI  discusses  this  change. 

Department  of  Defense 

As  a  result  of  the  Blue  Ribbon  Defense  Panel  Report  of  July  1970,  a 
Deputy  Director  (Test  and  Evaluation)  was  established.  It  was  the 
responsibility  of  this  new  office  to  provide  test  and  evaluation 
direction  as  well  as  to  review  and  approve  test  plans  for  major  systems. 
More  specifically,  the  first  and  incumbent  Deputy  Director,  LTG  Alfred 
D.  Starbird,  USA  (Ret.),  was  to: 

— Recommend  directly  to  the  Secretary  of  Defense  any  changes 
required  in  Test  and  Evaluation  policies  and  procedures. 
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— Monitor  closely  all  major  acquisition  programs,  advising  the 
DSA1C  and  Secretary  of  Defense  directly  at  key  decision  points  as 
to  the  adequacy  of  the  accomplished  and  planned  test  and  evaluation. 
—Initiate  and  coordinate  accomplishment  of  such  joint  testing 

as  necessary. 

— Oversee  all  major  Department  of  Defense  ranges  and  test  facilities. 
This  relatively  small  DOD  Test  and  Evaluation  Office  consists  of 
15  military  and  18  civilian  personnel  divided  into  three  assistant 
directorates  as  shown  in  figure  five. 


PIG.  5,  DDR&E  TEST  AND  RESOURCES 


Although  few  in  number,  the  Impact  of  this  organization  on  test  and 
evaluation  (particularly  operational  testing)  has  been  immense.  As 
LTG  Starbird  recently  pointed  out,  "By  far  the  most  significant  accom¬ 
plishment  to  date  is  that  all  in  DOD  engaged  in  .ys terns  acquisition 
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now  understand  what  is  wsnted  in  test  and  evaluation  and  are  attempting 
to  provide  it."  As  this  quotation  lapllas,  SOD  is  primarily  interested 
in  materiel  acquisition;  that  is,  devolopnental  and  operational  teats. 
To  data,  it  has  shown  vary  little  interest  in  the  Amy's  VOTE  of 
organisation,  doctrine,  and  tactics. 

Office  of  the  Chief  of  Research  and  Developnent 
The  Test  and  Evaluation  Branch  of  the  Msnagenent  and  Test  Division 
of  OCSD  (figure  six)  has  as  a  prlaary  mission  that  of  aonitoring, 
managing  end  supervising  the  developmental  testing  program  of  the 
Army.  Some  consider  them  to  be  the  DA  "proponent"  for  DT.  This 
branch,  consisting  of  approximately  10  people  also  coordinates 
operational  testing  with  ACS FOR. 
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FIG.  6,  OFFICE  CF  THE  CHIEF  OF  RESEARCH  AND  DEVELOPMENT 
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Their  efforts  entail  Insuring  the  availability  of  developmental 
iteas  for  testing*  as  wall  as  identifying  the  resource  and  test  support 
required.  They  also  review  developaental  allestones  to  insure  test 
adequacy  and  conformity  with  governing  regulations  and  interface  the 
developaental  effort  with  OTEA. 

In  short*  the  branch  aanages  all  developaental  testing  done  by 
the  aateriel  developer  and  coordinates  closely  with  the  operational 
testing  agencies. 

Asssitant  Chief  of  Staff  for  Force  Development 

The  Assistant  Chief  of  Staff  for  Force  Development  has  DA  staff 
responsibility  for  all  Army  Operational  Testing  and  Force  Development 
Testing  and  Experimentation.  Prior  to  1973,  the  staff  itself  was 
organized  to  manage  these  user  tests;  however*  with  the  fo.u.ation 
of  OTEA,  this  function  was  transferred  to  that  field  agency. 

Currently  OTEA  functions  as  a  field  operating  agency  under  the 
jurisdiction  of  the  ACS FOR.  Additionally,  the  Commanding  General, 

OTEA,  is  designated  the  Deputy  Assistant  Chief  of  Staff  for  Force 
Development  for  User  Testing,  and  in  compliance  with  DOD  policy, 
reports  to  the  Chief  of  Staff  of  the  Army  throiugh  the  ACSFOR.  The 
organization  and  functions  of  OTEA  will  be  examined  in  detail  later 
in  this  chapter. 

The  Test  and  Evaluation  Command 

The  Test  and  Evaluation  Command  is  an  integral  part  of  the  Army 
Materiel  Comnand.  As  one  of  the  nine  subordinate  commands  of  AMC, 
TECOM  conducts  all  Army  developmental  tests  for  proposeo  weapons 
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systems  sad  aster lei.  In  addition,  they  provide  customer  testing 
service  for  the  Commodity  Cn— ends,  private  Industry  and  other 
Government  agencies. 

To  accomplish  its  mission,  TEGOM  has  14  installations  and  activities 
located  across  the  United  States  and  in  Alaska  and  the  Canal  Zone. 
Included  among  these  are  the  six  branch  test  boards;  the  six  proving 
grounds  and  ranges;  and  the  two  environmental  test  centers  as  shown 
in  figure  seven. 

The  developaental  test  nay  consist  of  two  phases:  the  engineering 
test  phase,  generally  conducted  at  the  proving  grounds  and  environ¬ 
mental  test  centers;  and  the  service  test  phase,  usually  conducted 
at  the  boards. 

The  bulk  of  the  Army  personnel  involved  in  test  and  evaluation 
are  assigned  to  TEGOM.  Although  reduced  in  strength  by  one-third 
since  its  activation  in  1962,  the  command  still  maintains  a  strength 
of  14,213.  Approximately  30  percent  of  this  total  is  military. 

The  Operational  Test  and  Evaluation  Agency 

In  September  1972,  the  Secretary  of  the  Army  directed  the  activa¬ 
tion  of  the  US  Army  Operational  Test  and  Evaluation  Agency  under  the 
DA  staff  supervision  of  ACSFOR  and  located  at  Ft  Belvoir,  VA.  The 
mission  of  OTEA  is  to  manage  nil  User  testing.  Specifically,  it  is 
to: 

— Plan,  direct  and  evaluate  Operational  Testing  of  all  major 
and  selected  non-major  systems. 

— Coordinate  testing  for  other  non-major  systems. 
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7,  TECOM  Installation* 
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— Manage  aajor  and  coordinate  non-major  Force  Development  Test 
and  Exper lata tat Ion. 

— Coordinate  Army  participation  In  the  planning  for  and  the 
conduct  of  Joint  Osar  Testing. 

— Provide  a  strong  focal  point  for  Army  operational  testing. 

OTEA  is  organized  along  functional  lines  with  seven  divisions 
(figure  el^it).  The  Test  Design  Division  plans  the  test  which  is 
conducted  in  the  field  by  troop  units  of  FOSS COM  under  the  guidance 
of  a  five  nan  eleasnt  from  the  Field  Test  Division.  The  Operational 
Support  Division  insures  that  all  support  necessary  is  available  for 
the  actual  conduct  of  tests  while  the  Technical  Support  Division  pro¬ 
vides  professional  civilian  talent.  Data  obtained  from  tests  are 
reviewed  by  the  Evaluation  Division  and  final  reports  are  prepared 
using  test  assessments  and  other  material  available. 

OTEA  also  prepared  the  Five  Year  Test  Program  (FYTP)  which  includes 
major  OT  and  FTDE.  The  agency  is  authorized  a  total  strength  of  some 
250. 
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TEST  DESIGN- 
DIVISION 


TECHNICAL 

SUPPORT 

DIVISION 


FIELD  TEST 
DIVISION 


PERSONNEL  AND 

ADMINISTRATION 

DIVISION 


EVALUATION 

DIVISION 


PLANS  AND 

OPERATIONS 

DIVISION 


OPERATIONAL 

SUPPORT 

DIVISION 


FIG.  8,  OPERATIONAL  TEST  AND  EVALUATION  AGENCY 


Training  and  Doctrine  Con— nd 


Within  the  Training  and  Doctrine  Co— nd,  a  Deputy  Chief  of  Staff 
for  Coobat  Developments,  among  other  duties,  handles  testing  respon¬ 
sibilities  for  the  organisation.  To  assist  him  in  this  function,  a 
14-nan  Exper inentat ion  and  Test  Division  has  been  formed.  Subelements 
of  this  division  are  as  shown  in  Figure  Nine. 


FIG.  9,  EXPERIMENTATION  AND  TEST  DIVISION,  TRADOC 
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Principal  tenting  functions  of  TKADOC  Include: 


— Manageaent  of  operational  testing  for  non-major  systems  other 
then  those  selected  for  OTEA  control. 

— Coemsnd  of  the  Coabet  Darelopments  Experimentation  Command. 

— Origination  of  the  bulk  *f  force  development  tests  and 
experiments .  1 

In  order  to  accomplish  the  awigned  OT,  TRADOC  normally  delegates 
test  responsibilities  to  the  functional  centers  (Coabat  Arms  at  Fort 
Leavenworth,  Administration  at  Ft^larrison  and  Logistics  at  Ft  Lee). 
In  turn,  these  functional  centers  ’'ass  these  tests  to  the  appropriate 
school  for  planning  and  execution.^ 
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Fore*  Development  Tests  sad  Experiments  origins ting  st  TRADOC 
or  tbs  schools  may  bo  assigned  directly  to  CMC  or,  if  appropriate , 
passed  to  ACSYOft  for  execution  by  MASSTER. 

Coabet  Developments  Experimentation  Commend 

The  Combat  Developments  Experimentation  Pound  was  formed  In 
1956  and  is  currently  headquartered  at  Ft  Ord,  California,  with  its 
field  test  facility  at  Hunter  Liggett  Military  Reservation.  Fro*  its 
inception,  CMC  has  specialised  in  scientific  Force  Development 
Testing  specifically  field  experiments.  Its  mission  calls  for  it 
to: 

— Develop  and  provide  experimentally  derived  data  as  input  for 
models,  simulations  and  var  games  of  the  various  coabet  development 
agencies . 

— As  directed,  to  test,  analyse  and  provide  experimentally — derived 
data  on  developmental  options. 

--To  verify  reconamnded  solutions  for  operational  concepts,  materiel 
requirements  and  organisation  structure  through  field  experimentation. 

The  Experimentation  Command  is  authorized  2488  military  and  76  DA 
civilian  personnel.  In  addition,  CDEC  is  augmented  by  a  contract 
scientific  support  laboratory  consisting  of  over  100  scientists  and 
their  administrative  assistants.  The  organisation  of  CDEC  is  shown 
in  figure  ten. 

The  Experimentation  Group  comprised  of  two  infantry  companies 
an  armor  company,  and  a  transportation  coiqpany  along  with  engineer, 
maintenance  and  Instrumentation  elements,  account  for  75  percent  of  the 


total  CMC  strength.  Iha  Majority  of  the  required  teat  player  and 
support  troops  are  provided  by  thla  unit*  while  POtSGOM  supplies 


personnel  with  occupational  specialtiee  not  available  within  the 


group. 


DCS  “ERSCNNEL 


DCS  LOGISTICS 


DCS  EXPERIMENTATION 


DCS  INSTRUMENTATION 


DCS  PROGRAMS  AND 
PROJECT  ANALYSIS 


DCS  MANAGEMENT 
AND  BUDGET 


PIG.  10,  COMBAT  DEVELOPMENTS  EXPERIMENTATION  COMMAND 
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Sun—ry 

In  su— ary,  we  see  that  at  tha  Depart— at  of  Aragr  lml,  OCtO  haa 
responsibility  of  DT  while  ACSIQI  —nay  tha  Soar  Tests.  TEOOH 
actually  conducts,  with  tha  assistance  of  its  boards,  tha  Develop— ntal 
tasts.  Tha  — ra  laportent  of  the  Ota  are  conducted  by  OTEA  with  TKADOC 
and  tha  schools  — a— Inf  responsibility  of  all  reaainlng  OTs.  In  the 
field  of  VOTE,  both  GDEC  and  MASSTEft  conduct  tiailar  tests;  however, 

CDEC  Is  co— ended  by  TIADOC  and  MASSTEA  by  FDISCOM.  All-in-all,  some 
19,000  individuals  are  directly  Involved  in  the  Army  Test  and  Evaluation 
Procesa. 
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The  rn— nrilm  General  of  III  Corps  located  at  Ft  Hood  also  serves 
as  the  fn— an  rllin  General  of  MAS  STEF.  Since  MASSTEt  does  not  possess 
dedicated  test  troops*  the  two  divisions  of  the  corps  are  called  upon 
to  actively  support  teste.  This  "dual-bat"  arrangement  facilitates 
testing  and  allows  for  the  resolution  of  support  problems  at  the 
lowest  level.  Figure  eleven  shows  the  MASSTEE  organisation. 


MASSTE8 

COMBAT  SUPPORT 
DIRECTORATE 

“I 

DCS  PERSONNEL  AND 
ADMINISTRATION 

- 

GROUND  SUPPORT 
DIRECTORATE 

DCS  MANAGEMENT 

AND  BUDGET 

AIR  COMBAT 
DIRECTORATE 

DCS  OPERATIONS 

AND  PLANS 

ENGINEERING  AND 
INSTRUMENTATION 
DIRECTORATE 

DCS  LOGISTICS 

*• 

COMMAND  CONTROL 

AND  COMMUNICATIONS 
DIRECTORATE 

FIG.  11*  MODERN  ARMY  SELECTED"  SYSTEM 
TEST,  EVALUATION -AND  REVIEW 
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CKAPTSK  V 


FUIUU  HST  EEQUIKtBRTS 
Central 

The  purpose  of  this  chapter  is  to  discuss  the  foreseeable  test 
requirements  of  the  Army  as  it  Is  with  those  requi resents  that  any 
test  systes  suet  cope.  In  specifying- teat  requireaents ,  the  "further- 
tbe-fux*ier"  rule  applias.  While  near  tars  requireaents  are  normally 
veil  known,  add  and  long-range  requireaents  are  subject  to  the  normal 
vagaries  of  events.  It  is,  nevertheless,  possible  to  provide  estimates 
that  place  this  study  in  sharper  perspective. 

The  principle  source  document  that  detail  *lth  considerable 
accuracy  the  test  load  for  major  Operational  Tests  and  all  Force 
Developaental  Test  and  Experiments  is  the  previously  discussed  Five- 
Year  Test  Frograa.  Omitted  from  the  FYTP  are  the  numerous  tests 
conducted  by  AMC  and  the  minor  OT’s.  Even  though  it  encompasses  only 
ten  percent  of  the  total  Army  test  requirement,  it  does  include 
all  aajor  Army  material  systems  scheduled  for  testing. 

Operational  Tests 

Perhaps  the  most  Important  tests  facing  the  Army  during  the  next 
decade  are  those  designated  as  Operational  Tests  of  major  and  selected 
non-major  equipment.  Included  in  this  category  are  such  high 
dollar  items  as  the  Advanced  Attack  Helicopter,  the  Heavy  Lift  Heli¬ 
copter,  the  XM1  Tank,  and  the  new  Infantry  Combat  Vehicle,  to  mention 
a  few.  A  list  of  the  52  systems  currently  requiring  the  full  range 
of  Operational  Tests  is  shown  at  figure  twelve. 
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MAJOR  SYSTEMS 

AAH 

STINGER 

AN/TSQ-73 

IMP  8"  NUC  PROJ 

ARSV 

MICV 

AJSM 

LT  WT  CO  MORTAR 

NAVSTAR 

DRAGON 

ATMAC  &  DDL 

MICV  FPW 

ARTY  LOG  RADAR 

LANCS 

AN/TTC-39 

MORT  LOC  RADAR 

CUSP 

XM-1 

CEFLY  LANCER 

M60A1  PIP 

IMP  HAWK 

PERSHING  II 

DSCS 

PER  ARMOR 

HLH 

SAFEGUARD 

FAMECE 

REMBASS 

HELLFIRE 

SITE  DEF 

FUSE  XM587 

RPV/DRONES 

SHORAD 

SAM-D 

XM204 

SCAT  MINES 

XM-198 

TACFIRE 

8 "HOW  M11PE2 

SEAS 

TRI-TAC 

TOS 

UN  ENG  TRAC 

SQD  AUTO  WPN 

ASH 

UTTAS 

TOW  NIGHT  SIGHT 

TACSATCOM 

VRFWS 

M60A2 

TILT  ROTOR 

COBRA  TOW 

Fig*  12*  Major  and  Selected  Non 'major  systems 
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Looking  at  the  test  load  another  way,  figure  thirteen  depicts 
how  *hese  tests  are  scheduled  over  time. 

A  second  category  of  Operational  Tests*  the  so  celled  "other  OT's," 
are  OT's  for  less  important  systems  such  as  generators,  1R  viewers, 
radios,  certain  radars,  and  night  vision  goggles.  Figure  thirteen 
also  presents  these  test  requirements  over  time.  In  excess  of  ISO 
tests  of  this  type  have  been  identified. 

Developmental  Tests 

By  far,  the  bulk  of  Army  testing  is  done  by  TECOM.  In  addition 
to  the  conduct  of  allJDI's,  TECOM  performs  numerous  tests  for  Project 
Managers,  Commodity  Commanders,  and  contractors  as  well  as  conducting 
"inhouse"  testing.  Nearly  801  of  dedicated  Army  testing  manpower 
is  located  in  TECOM.  For  the  purpose  of  this  study,  it  is  sufficient 
to  use  the  2nd  quarter  of  FY  1974  to  indicate  t'.ie  enormous  test  work¬ 
load.  During  that  period,  TECOM  had  active  either  in  the  planning, 
executing,  or  reporting  stage  over  2000  tests.  Of  these,  some  1200 
or  602  were  "customer"  tests,  i.e.,  those  performed  for  specific  hard¬ 
ware  developers  and  20  percent  could  be  classified  as  development 
tests. 

Force  Development  Test  and  Experimentation 

The  FDTE  load  has  historically  been  the  least  predictable,  for 
these  tests  are  tied  strictly  to  ideas  and  are  not  tied  to  the  material 
acquisition  budget  cycle.  The  term  FDTE  brings  under  an  umbrella 
such  time-worn  expressions  as  "Field  Experiment,"  "Troop  Test,"  "Field 
Test,"  and  "Field  Evaluation." 
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Examples  of  presently  scheduled  FDTE's  Include: 

—Division  Staff  Organization  and  Procedures. 

— Airspace  Control. 

—Attack  Helicopter  Daylight  Offense. 

— Army  Combat  Battalion  Close  Air  Support  Training. 

— Vulnerability  of  Designator  Equipped  forward  Observer. 

There  are  70  such  FDTE's  required  over  the  next  6  years,  with 
their  tirtefram  as  shown  in  figure  thirteen.  This  rather  unusual 
schedule  is  not  surprising:  Some  tests  will  slip  and  many  new  ones 
will  be  scheduled  as  out-year  resource  availability  becomes  more 
evident.  It  does  point  out,  however,  the  necessity  for  more 
innovative  techniques  of  scheduling  and  the  need  for  better  overall 
management  of  the  FDTE  program. 


It  can  be  seen  froa  the  proceeding  paragraphs  that  the  known 
requlreaents  for  Army  teats  are  1— anae.  One  to  two  year  requirements 
are  fairly  firm  and  it  is  reaaonable  to  expect  that  the  future  annual 
Army  requirement  for  tests  will  remain  around  the  125  level  for  "User" 


tests  and,  based  or  information  received  from  TECOM,  around  the 
1200-1500  level  for  UK  tests. 
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CHAPTER  VI 


ONGOING  ACTIONS  BEARING  OH  THE  TEST 
AND  EVALUATION  PROCESS 


DA  Reorganization 

On  4  January  1974,  a  decision  was  made  by  the  Chief  of  Staff  to 
reorganize  and  streamline  the  Army  staff.  The  principal  objectives 
for  this  reorganization  were: 

— Improve  the  direction  and  control  of  the  staff. 

— Eliminate  fragmentation  of  responsibilities. 

— Remove  staff  layering. 

— Establish  clear  responsibility  for  the  key  functions  of 
people,  dollars,  planning,  materiel  acquisition  and  logistics.  The 
reorganization  was  effective  on  20  May  1974.  Up  to  that  date,  the 
DA  staff  was  as  shown  at  figure  fourteen.  The  new  organization  is 
shown  at  figure  fifteen. 

This  new  structure  caused  some  major  shifts  of  responsibility 
for  test  and  evaluation.  Effective  with  the  reorganization;  respon¬ 
sibilities  for  test  management,  previously  under  the  Assistant  Chief 
of  Staff  for  Force  Development  and  the  Chief  of  Research  and  Develop¬ 
ment,  were  transferred  to  the  Operational  Test  and  Evaluation 
Agency,  the  Deputy  Chief  of  Staff  Operations  and  Plans  (DCSOPSli ,  ar.u 
the  Deputy  Chief  of  Staff  Research,  Development,  and  Acquisition  (DCSRDA) 

The  new  organization  established  OTEA  as  a  Field  Operating  Agency 
directly  under  the  Chief  of  Staff,  responsible  for  the  overall  manage¬ 
ment  of  User  Testing.  The  management  of  Development  Testing  was 
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rig.  14.  Current  Arm/  Staff  Organisation 


)./ REPORTS  DIRECTLY  TO  THE  SA  AliD  CSA  AS  A  PERSONAL  STAFF  OFFICER. 

2/  ALSO  UNDER  THE  DCSPER  FOR  DESIGNATED  FU'JCTIONS- 

?  AiSO  REPORT  OIRECTIY  TO  THE  CSA  AS  PERSONAL  STAFF  OFFICERS  TOR  DESIGNATED  FUNCTIONS. 
i'  REPORTS  DIRECTLY  TO  THE  SA  FOR  CIVIL  FUNCTIONS . 

5  ALSO  CHIEF  OF  PUBLIC  INFORMATION.  REPORTING  DIRECTLY  TO  THE  SA, _ 


PiR*  1».  fMtur*  Amy  Staff  organisation 
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assigned  to  the  DCSm  •  Currently*  there  is  still  some  discussion 
concerning  the  division  of  responsibility  for  User  Testing  between 
OTEA  end  DC  SOPS.  The  directories  responsible  for  test  and  evaluation 
within  DCS IDA  and  DCSOPS  are  shown  at  figures  sixteen  and  seventeen. 

Amy  Materiel  Acquisition  Review  >Committee  Study 
In  Noveaber  1973*  the  Secretary  of  the  Any  directed  that  an 
independent  review  be  aade  of  the  Amy's  total  materiel  acquisition 
process.  On  6  December  1973,  e  memorandum*  signed  by  the  Under 
Secretary  and  the  Vice  Chief  of  Staff;  was  given  to  Dr.  Wendell  Sell 
requesting  that  he  heed  the  Amy  Materiel  Acquisition  Review  Committee 
(AMABC) .  A  separate  team  was  organized  to  conduct  that  portion  of 
the  study  dealing  with  test  and  evaluation. 

Guidance  to  the  AMARC  included  a  list  of  issues  to  be  addressed; 
however,  the  coamittea  was  free  to  consider  other  issues,  if  warranted. 
The  directed  issues  that  concerned  the  test  and  evaluation  system 
were: 

--Where  can  personnel  reductions  best  be  made? 

— Should  the  Amy  have  a  separate  command  for  testing? 

—Are  Test  boards  needed?  If  so,  how  many  and  what  should 
their  functions  be? 

—What  should  be  the  reporting  chain  for  the  various  test 
activities? 

— How  much  of  proving  ground,  range  and  similar  test  activity 
operations  can  be  by  contract? 

—Can  we  close  some  of  our  test  facilities?  If  so,  which? 
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16,  Functions  and  Organization, 
Materiel  Plans  and  Programs 
Directorate,  DCSRDA 


DEPUTY  CHIEF  OF  STAFF 
OPERATIONS  AND  PLANS 


l 


REQUIREMENTS 

DIRECTORATE 


•  Operational  Concepts 

•  Material  Requirements  and 

Priorities 

e  cembat  organization 

•  Test  and  Evaluation 

e  Force  Development  Policy 
and  Guidance 


Fig,  17,  Function*  and  Organization 
Requirements  Directorate, 
DCSOPS 


On  2  April  1974,  the  AHARC  study  vss  released  to  the  Any  Staff 
for  comment.  One  full  chapter  was  davoted  to  the  Testing  Team  Report 
In  addition  to  the  six  directed  issues,  the  testing  teaa  examined 
soae  12  additional  issues  concerning  Any  nesting  and  evaluation. 
These  additional  questions  fall  generally  into  six  areas: 

--Independent  test  evaluation. 

— The  organization  for  DT  and  OT. 

— FDTE  eaphasis. 

— Testing  personnel  qualifications. 

— Discretionary  testing  programs. 

— Testing  facilities. 

During  its  review,  the  Testing  Team  found  that  if  it  focused  its 
review  on  the  six  areas  identified  above,  the  issues  contained  in  the 
Under  Secretary's  memo  would  also  be  addressed.  The  team  found  that 
the  Army,  in  striving  to  maintain  OT  independent  from  DT,  was  in 
fact  injecting  unnecessary  costs  into  the  test  system.  According 
to  the  AMARC  study,  there  exists  a  significant  overlap  between  OT 
and  the  service-use  phase  of  DT.  The  team  found  that  developers, 
users,  and  testers  all  expressed  considerable  difficulty  in  sorting 
out  the  proper  division  of  testing  responsibilities  between  DT  and 
OT.  Another  major  finding  was  that  OT  and  DT  were  sufficiently 
different  to  justify  separate  facilities  and  organizations  and  that 
AMC  must  have  a  capability  to  accomplish  DT  as  part  of  its  materiel 
acquisition  process. 

As  a  result  of  their  findings,  the  Testing  Team  arrived  at  the 


following  recommendations: 


— Designate  a  single  Army  Staff  element  to  monitor  the  total 
acquisition  process. 

—Present  independent  DT  and  OT  evaluations  at  IPR/ASARC 
meetings. 

— Emphasis  the  difference  between  DT  and  OT,  based  primarily 
on  the  technical  orientation  of  DT  and  the  operational  orientation 
of  or. 

—Do  not  create  a  major  testing  command  to  accomplish  developer 
and  user  testing  separate  from  the  materiel  developer  or  the  combat 
developer. 

— The  emphasis  on  the  separation  of  OT  and  DT  should  be  changed 
from  separate  testing  to  independence  of  design  and  evaluation  to 
permit  more  efficient  use  of  testing  resources  applied  to  integrated 
or  combined  tests. 

— Enhance  TRADOC  FDTE  and  OT  capabilities  by  assigning  it 
MASSTER,  the  Service  Test  Boards,  and  an  additional  analytical  capa¬ 
bility. 

— Assure  independence  of  DT  design  and  DT  evaluation  by 
assigning  control  of  both  functions  to  the  Army  Materiel  Systems 
Analysis  Agency  leaving  TECOM  as  a  testing  service. 

— Review  existing  activities  to  reduce  costs  by  consolidation, 
closures,  or  increased  contractor  support. 

— Place  OTEA  directly  subordinate  to  the  Chief  of  Staff. 

— Consider  placing  Dugway  Proving  Ground  in  standby  status. 
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— Enhance  personnel  capabilities  by  expanding  career  develop¬ 
ment  opportunities,  and  Increasing  use  and  duration  of  current  stabili¬ 
zation  progress . 

— Institute  discretionary  testing  progress  (approximately  five 
percent  of  budgets)  to  foster  low  cost,  high  payoff  initiatives. 

— Modify  current  test  report  and  evaluation  procedures  to 
Include  individual  opinions  of  knowledgeable  personnel  in  test 
report s/evaluations  furnished  decisionaakers . 

Summary 

The  impact  of  the  reorganization  and  the  AMARC  study  were  considered 
in  our  evaluation  as  both  have  caused  significant  changes  in  the  present 
test  and  evaluation  process. 
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CHAPTER  VII 


EVALUATION 

General 

This  chapter  provides  the  heart  of  the  report.  While  Chapters 
I  through  VI  laid  the  explanatory  framework  necessary  for  under¬ 
standing  the  present  test  and  evaluation  system,  herein  is  contained 
the  evaluation  of  that  system,  including  our  findings  and  recommenda¬ 
tions.  The  aims  of  the  study  were  to: 

—Identify  problems. 

—Develop  an  improved  system. 

— Recommend  procedures  and  assign  responsibilities  to  achieve 
the  improved  system. 

Eight  major  problem  areas  have  been  identified  as  being  sufficiently 
significant  to  be  considered  for  substantive  change.  These  are: 

— Methods  vs  Goals  of  Test. 

— Philosophy  of  FDTE, 

— Fragmentation  of  FDTE. 

— Interpretation  of  DOD  Policy. 

— The  Coordinated  Test  Program. 

— DA  Staff  Responsibility. 

— Fragmentation  of  OT. 

— Troop  Involvement  in  DT. 

From  a  format  standpoint,  each  of  the  eight  areas  are  addressed 
in  separate  sections.  Each  section  is  preceeded  by  the  principal 
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finding,  followed  by  *  discussion  of  that  finding.  Included  in  the 
discussion  is  the  arguasnt  for  whet  we  believe  to  be  an  appropriate 
solution  to  the  identified  problem.  The  section  concludes  with  a 
specific  recoaaendation  cr  series  or  recoanendations  for  action. 
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FINDING 


Methods  vs  Goals  of  Tests 

Test  goals  are  often  mixed  with  test  methods,  one  being  defined 
in  terms  of  the  other.  This  leads  to  confusion  and  misunderstanding 
and  is  due,  in  part,  to  unclear  regulations  and,  in  part,  to  lack 
of  understanding  of  the  philosophy  of  testing. 


Discussion 


It  became  evident  during  the  course  of  our  study  that  considerable 

confusion  existed  concerning  various  test  goals  and  various  test 

methods  to  achieve  those  goals.  There  seems  to  be  a  penchant  for 

either  over-defining — hence  restricting — or  under-defining — hence 

confusing — many  of  the  terms  and  definitions  in  the  field  of  test 

and  evaluation.  The  basis  for  this  finding  stems  from  numerous 

discussions,  the  contents  of  current  regulations  and  the  personal 

experience  of  the  study  group.  On  the  surface,  the  distinction 

between  methods  and  goals  of  tests  may  seem  trivial.  We  hold,  however, 

that  the  point  is  vital.  Without  clear  definitions  as  a  guidepost, 

it  is  easy  to  become  lost.  The  confusion  is  most  evident  in  the 

FDTE  area.  Consider  the  following  extract  from  AR  10-4: 

Force  Development  Testing  and  Experimentation 
Tests,  ranging  from  the  small  in  scope,  highly 
instrumented,  high  resolution  field  experiment 
to  a  broader  in  scope,  less  instrumented,  low 
resolution  and  highly  subjective  field  test,  that 
are  performed  to  support  the  force  development 
process  by  examining  the  impact,  potential,  and 
effectiveness  of  selected  concepts,  tactics, 
doctrine,  organization,  and  materiel. 
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Here  we  aee  goal  aad  Method  luaped  together  in  one  stew,  with 

neither  having  aufficlent  clarity  to  be  of  much  help.  Even  digging 

deeper  into  the  only  other  reference  source  defining  FETE — AR  71-3 

(still  in  draft)-- we  find  the  following: 

Tests,  ranging  froa  the  saall  in  scope,  highly 
instruBented,  high  resolution  field  experiment, 
to  the  broader  in  scope,  less  lnstruaented,  low 
instruaented,  low  resolution  field  test.  Data 
from  the  tests  are  evaluated  largely  by  subjective 
rather  than  analytical  techniques.  These  tests  are 
perforaed  to  support  the  force  development  process 
by  examining  the  inpect,  potential,  and  effectiveness 
of  selected  concepts,  tactics,  doctrine,  organization, 
and  materiel.  FDTE  assesses  the  interdependence 
among  doctrine,  tactics,  organization,  and  materiel; 
further,  FDTE  may  support  the  materiel  acquisition 
process  by  providing  data  to  assist  ’.n  the  estab¬ 
lishment  of  the  Required  Operational  Capability  (ROC) , 
to  develop  fundamental  data  necessary  for  a  full 
understanding  of  the  performance  of  a  materiel 
system,  or  to  assist  in  validating  doctrine  and/or 
tactics  to  counter  threat  response  to  a  system  once 
deployed.  (Underlines  added.) 

Again,  we  observe  goals  and  methods  intertwined.  Moreover,  tha 
first  underlined  statement  is  not  true.  With  regard  to  the  last 
underlined  statement,  an  FDTE  can  just  as  easily  validate  doctrine 
and/or  tactics  that  are  simply  better  than  present  ones  as  well  as 
those  designed  to  counter  a  threat  response. 

Goals 


The  goals  of  Army  tests  will  be  covered  in  detail  in  sections 
that  follow.  At  this  juncture,  it  is  sufficient  to  indicate  the 
following  goals  that  cover  virtually  all  Army  tests: 
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--Provide  inforuation  concerning  material  performance. 

—Provide  information  concerning  material-soldier  performance. 

—Provide  information  concerning  material-soldier-mission 
performance. 

— Provide  information  concerning  material  concept  validity. 

— Provide  information  concerning  organization,  doctrine, 
tactics  validity. 

The  first  two  are  DT's,  the  third  is  an  OT,  while  the  last  two 
are  FDTE's. 

To  achieve  any  of  the  golas ,  there  are  a  variety  of  test  methods 
available.  The  method  selected  for  conducting  a  test  will  vary  as 
necessary  to  meet  the  specific  information  desired  and  to  meet  certain 
operational  constraints  faced  in  the  real  world.  Across  the  continuum 
of  test  methods  there  are  two  extremes:  the  field  experiment  and  the 
field  evaluation.  The  former  is  classically  scientific  (a  hypothesis, 
chance  elements  removed,  repeatable,  etc.)  while  the  latter  is,  in 
essence,  informed  Judgment.  In  some  middle  ground  there  is  the  field 
trial  method,  which  incorporates  elements  of  both.  Figure  eighteen 
illustrates  the  test  method  continuum  and  contains  eight  general 
characteristics  for  each  of  the  three  types.  Note  that  an  experiment 
is  virtually  all  objective,  while  the  evaluation  is  essentially  all 
subjective.  As  in  any  generalization,  there  may  be  exceptions. 

The  term  "can  operationally  define  explicitly"  used  in  row  1 , 
column  1  simple  means  the  tester  or  test  requester  is  able  to  describe 
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I  Objectivity 


Subjectivity 


Objectivity 


Degree  of  objectivity 
'****■*• 

'  . . . " 

***»•», 

-•.^‘'“’Degree  of  «ub joctivity  ****•«*«»_ 


Subjectivity 


FIELD  .DCPERIMENT 

FIELD  TRIAL 

FIELD  EVALUATION 

DECISION 

Can  operationally 
define  explicitly 

Can  operationally 
define  segments 

Cannot  operationally 
define 

PROCEDURE 

Many  repl icationa. 
objective  analysis, 
measure  variables 

Few  replications 
objective  and 
judgmental  analysis 

No  planned  replica¬ 
tions,  long-term 
judgmental  analysis 

RESOLUTION 

Seconds -Minutes 

Hours-Days 

Day s-Weeks -Months 

DATA 

Objective,  through 
instrumented  physical 
measurement 

Sea  led  judgment 
and  objective 

Judgments l  and 
objective 

CONTROL 

Rigid 

Limited 

None 

ANALYSIS 

Statistical 

Evaluator  judgments 
suppumented  by 
statistical  summaries 

Evaluator  judgments 
and  tabular  data 

TYPE  RESULTS 

Quantitative 

Quantitative/ 

qualitative 

Qualitative 

USE 

Reliable  data  for 
input  to  other 
analysis  or  decision. 
Can  draw  statistical 
inference 

Objective  and 
judgmental  evalua¬ 
tion  of  system. 

Judgmental  evaluation 
of  system 
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numerically  Che  operation  under  cone  ide  rati  on.  As  an  example,  the 
phrases,  "the  system  must  be  able  to  achieve  a  .90  kill”  or  "the 
system  must  transmit  75Z  of  all  messages  in  no  less  than  one  hour" 
precisely  define  what  is  expected  and  an  experiment  can  then  be 
designed  to  determine  if  the  system  can  achieve  these  objectives. 

Tests  should  ideally  be  conducted  as  pure  experiments.  The  proper 
experiment  removes  doubt  and  provides  firm  Information  upon  which 
to  base  a  decision.  There  are,  however,  reasons  why  an  experiment, 
cannot  be  run  and  some  other  method  of  test  must  be  substituted  in 
its  place.  Several  such  reasons  fellow: 

— Isposslble  to  operationally  define.  Many  tests  fit  this 
constraint.  While  tests  of  material  and  very  specific  tactical 
questions  may  usually  be  defined,  tests  of  organization,  broad  tactics 
and  doctrine  are  normally  too  complex  to  define  in  toto. 

— Lack  of  instrumentation.  Suitable  instrumentation  is 
required  to  conduct  an  experiment.  Oft  times,  such  instrumentation 
(generally  expensive)  is  not  available. 

— Low  budget.  Fund  availability  is  sadly  self-explanatory. 

— Time .  An  experiment  normally  takes  considerable  time  in 
planning,  designing  and  executing.  Decisions  sometimes  can't  wait. 

— Hardware  configuration.  In  tne  case  of  material,  if  the 
hardware  under  test  is  a  weak  representation  of  the  item,  it  seldom 
makes  sense  to  conduct  a  detailed  experiment. 
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— Troop  availability.  The  beet  designed  experiment  often 
is  not  run,  ss  the  cost  in  terns  of  troops  is  soaetlnes  prohibitive. 

— Test  expertise.  Defining  end  designing  an  experiment  is 
one  of  the  aoet  difficult  tasks  inrthe  cycle  of  military  research. 
The  required  expertise  is  often  lacking. 

The  preceeding  constraints  all  tend  to  drive  testing  to  the  sub¬ 
jective  method,  i.e.,  to  the  right  of  the  scale  in  figure  eighteen. 
The  aim,  however,  should  always  be  to  achieve  maximum  objectivity 
within  the  constraints. 

Summary.  In  sum,  we  believe  that  test  goals  and  test  methods  should 
be  clearly  defined  and  separated.  Once  the  goal  of  a  specific  test 
is  understood,  the  method  should  then  be-selected  based  on  the 
considerations  just  discussed. 

RECOMMENDATION 

Rewrite  AR  71-8,  Force  Development:  Array  Program  For  Test 
Evaluation,  to  include  separate  goals  and  methods  sections. 
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FINDING: 

Philosophy  of  FDTE 

Though  the  requirement  Is  valid  and  the  capability  exists,  FDTE 
is  not  receiving  sufficient  enphasls  due  to  vague  regulations  and 
loose  definitions . 


Discus a ion 

There  is  little  doubt  that  FDTE  is  of  vital  importance  to  the 
Amy.  The  AMARC  Study  acknowledged  this  point  specifically.  One 
need  only  consider  the  impact  of  the  McNair  efforts  in  the  late  30 's, 
the  work  of  the  Howse  Board  in  the  late  50 's /early  60 's  in  the  area 
of  air  nobility  and  the  aasslve  series  of  reserve  component  tests 
conducted  in  the  early  70 's  to  recognize  the  vital  role  Force  Develop¬ 
ment  Testing  plays  in  the  Army.  While  Force  Development  testing  has 
always  been  done  by  the  Army,  it  was  generally  done  on  an  ad  hoc 
basis  until  the  formation  of  CDEC,  and,  later  MASSTER.  Fortunately, 
the  Army  has  created  two  permanent  organizations  that  are  well 
staffed  to  carry  out  FDTE.  The  term  itself — FDTE— is  of  recent  origin. 
Previously,  force  development  testing  was  camouflaged  under  a  number 
of  methodological  terms  that  described  test  methods  such  as  Field 
Experiment,  Field  Test,  Field  Evaluation,  and  Troop  Test.  By 
creating  the  term  "FDTE,"  the  Army  vent  a  long  way  toward  properly 
defining  the  goal  of  force  testing.  It  is  our  view,  however,  that 
the  present  definition  is  too  broad  to  be  useful  to  the  planner  and 
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the  current  regulations  too  vague  to  be  effectively  implemented. 

Several  regulations  define  FDTE  and  the  full  definition — given  in 
the  pieceedlng  8 eel ton — will  not  be  repeated  here.  Generally,  the 
definition  says  the  purpose  of  FDTE  is  to  test  tactics,  doctrine, 
organization,  material.  The  definition  stops  at  that  point. 

It  appears  to  the  study  group  that  several  changes  could  provide 
significant  long-term  benefits. 

First,  ve  propose  a  slight  modification  to  the  present  definition 
as  follows: 

Force  Development  Tests  are  tests  that  are  performed  to 
support  the  force  development  process  by  examining  the  potential  of 
proposed  tectics,  doctrine,  organization,  and  material  concepts. 

These  tests  aid  in  the  material  acquisition  process  by  providing 
information  relevant  to  the  establishment  of  validation  of  a 
Required  Operational  Capability  and  assists  in  validating  or 
rejecting  new  ideas  concerning  tactics,  doctrine,  and  organization. 

We  believe  this  is  a  more  meaningful  purpose  statement.  It  also 
deletes  any  reference  to  the  method  of  conducting  the  test. 

Second,  we  propose  that  Force  Development  Testing  be  separated 
into  two  parts: 

— Force  Test — Material  (FT-M) 

— Force  Test — Structure  (FT-S) 

The  first  would  focus  on  a  hardware  concept-as  the  prime  test  objective, 
while  the  second  would  focus  on  a  force  structure  concept  in  either 
organization,  tactics,  or  doctrine.  The  definition  could  be  as  simple 
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—The  purpose  of  Cbe  FT-M  is  to  sid  the  material  acquisition 
process  by  providing  information  relevant  to  the  establishment  or 
validation  of  a  ROC. 

— The  purpose  of  the  FT-S  is  to  examine  the  validity  of  pro¬ 
posed  tactics,  doctrine,  end  organization. 

In  the  FT-M,  simulation  of  conceptual  hardware  would  be  used. 

In  the  FT-S,  the  equipaent  used  would  normally  be  that  found  in  TOE 
units,  as  the  answers  sought  wtJld  pertain  to  tactics,  doctrine,  or 
organizations.  The  evaluation  of  FT-S  results  could  lead  to  revised 
tactics,  doctrine,  or  organization  or  could  identify  materiel  gaps. 

It  is  useful  at  this  point  to  cite  some  examples  of  tests  recently 
conducted  to  illustrate  more  clearly  exactly  what  the  study  group 
envlsons  for  the  FT-M  and  FT-S.  In  order  will  be  described: 

— FT-S:  focus  on  tactics 

— Ft-S:  focus  on  doctrine 

— FT-S:  focus  on  organization 

— Ft-M:  focus  on  material  in  ROC  development 

FT-S  (Helicopter  Tactics) .  In  a  series  of  field  trials  evaluating 
the  potential  of  various  tactics  concerning  armed  helicopters,  various 
tactics  were  tested.  These  included  aspects  of  scout  helicopter 
tactics,  low-level  flight  tactics,  massed  fire  tactics,  air  defense 
suppression  tactics,  night  tactics  and  point  target  attack  tactics. 

Through  these  tests,  new  tactics  involving  the  armed  helicopter 
have  evolved. 
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FT-S  (Army  Airspace  Control  Doctrine).  In  a  teat  involving  a  command 
poet  exercise,  the  problem  of  fundamental  doctrine  for  airspace 
control  was  addressed.  Three  separate  doctrinal  issues  were  con¬ 
sidered  and  major  conclusions  such  as  "There  is  no- requirement  for 
the  Army  to  regulate  Air  Force  traffic"  and  "Commanders  should  use 
a  'weapon  hold'  control  status  for  the  BEDEYE  as  normal  operating 
procedure"  were  reached. 

FT-S  (Division  Staff  Organization).  The  purpose  of  this  test  was  to 
investigate  the  staffing  of  the  division.  In  addition  to  many  minor 
changes — G1  and  AG  functions  combined,  liaison  officers  under  control 
of  operation  officer,  etc.— a  significant  change  in  the  overall  struc¬ 
ture  was  recommended;  namely,  abolish  the  Chief  of  Staff  and  the 
four  traditional  staff  sections,  and  assign  the  two  Assistant 
Division  Commanders  responsibility— one  for  operations  and  one  for 
support. 

FT-M  (Unmanned  Aerial  Surveillance  Material  Concept) .  This  test  is 
an  excellent  example  of  the  use  of  a  siau5 »ted  piece  of  hardware  to 
evaluate  a  material  concept.  The  purpose  of  the  test  was  "to 
evaluate  the  UASS  concept  to  determine  the  validity  of  the  material 
requirement"  through  the  use  of  specially  configured  drones.  The 
report  concluded  "that  an  unmanned  aerial  surveillance  system  is  not 
suitable  to  fulfill  the  requirement."  As  such,  the  test  provided 
important  data  in  a  negative  sense. 


113. 


72 


FT-M  (Motor  bike  Concept)*  Using  comer  cl  ally  available  trail  motor¬ 
bikes,  this  test  evaluated  the  potential  for  motorbikes.  As  a  result 
of  this  test,  it  was  determined  that  adding  motorbikes  to  reconnais¬ 
sance  units  would  enhance  unit  mission  capabilities  significantly. 

A  ROC  was  proposed  as  a  consequence  of  the  test. 

Summary.  Figures  nineteen  and  twenty  summarize  our  concept  graphically. 
In  figure  nineteen,  the  four  categories  of  Army  tests  are  displayed. 

The  outer  blocks  represent  the  entire  test  while  the  inner  blocks 
represents  the  relative  Importance  of  the  material  to  the  test.  In 
the  FT-S,  the  hardware  is  of  little  importance.  In  the  FT-M,  a 
material  concept  of  a  piece  of  hardware  is  evaluated  in  terms  of 
need,  potential  performance  and  into  what  organization,  for  what 
mission,  using  what  tactics,  it  might  be  employed.  The  DT  objectives 
are  almost  entirely  hardware  related,  while  OT  placed  the  material 
into  slightly  broader  perspective.  Figure  twenty  illustrates  the 
relationship  of  those  four  teat  types  to  one  another,  with  possible 
outcomes . 

Clearly  not  all  ideas  are  "testable"  ln-the  field,  nor  are  all 
ROC'S.  It  is  not  our  Intent  that  all  ROC's  and  all  ideas  be 
subjected  to  the  proposed  test  system — only  those  for  which  relevent 
and  useful  data  can  be  logically  derived  through  testing. 

RECOMMENDATION 

Adopt  the  proposed  changes.  Rewrite  AR  71-8  to  reflect  the  pro¬ 
posed  system. 
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Fig.  19,  Types  of  Tests 
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FINDING 


Fragmentation  of  FPTE 

There  ere  currently  two  organization*  within  the  Army ,  MASSTER 
and  CDEC,  specializing  in  potentially  duplicative  Force  Development 
Testing.  MASSTER  falls  under  command  of  FORSOOM  while  CDEC  answers 
to  TRADOC.  Neither  organization  is  adequately  controlled  or  directed 
by  higher  command. 


Discussion 

From  its  inception,  MASSTER  has  actively  sought  tests  in  various 
fields  of  interest.  It  has  often  acted  as  proponent  for  and  initiated 
testing  which  was  of  interest  to  its  staff.  This  well-intentioned 
enthusiasm  for  testing  has  resulted  in  some  criticism  as  to  test 
agency  impartiality. 

There  is  no  doubt  that  MASS TER' s  test  program  has  been  active  and 
far  ranging,  but  as  a  result  of  this  operational  latitude,  some  of 
its  tests  have  not  been  in  consonance  with  Army  needs. 

Because  of  the  suitable  terrain  and  availability  of  troops  at 
Fort  Hood,  MASSTER  has  often  directed  its  attention  toward  the  large 
unit  test  (battalion  and  higher).  Although  initially  limited  by  its 
lack  of  instrumentation  and  criticized  for  its  subjectivity,  an 
extensive  instrumentation  procurement  program  has  been  initiated  with¬ 
in  the  last  few  years. 
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On  the  other  hand,  CDEC  has  viewed  Itself  from  the  beginning  as 
a  highly  scientific  test  laboratory  only  Involved  in  the  rather 
restrictive  type  teats  known  as  experiments .  These  scientific 
experiments  have  Involved  snail  units  (company  and  below)  participating 
in  highly  instrumented  and  replicated  trials.  Test  design  at  CDEC 
is  tedious  and  tine- cons using.  As  a  result,  lead  times  for  testing 
are  extensive. 

Because  of  its  attention  to  detail,  numerous  replications,  and 
time  required  for  text  execution,  CDEC's  test  program  has  been  small. 

It  has,  therefore,  been  criticized  by  some  for  its  limited  output. 

In  addition,  CDEC  looks  upon  itself  aa  an  independent  test  agency 
totally  unaffected  by  the  result  of  the  test.  It,  therefore,  does 
not  solicit  or  initiate  tests  as  does  MASSTER. 

OTEA  has  teen  given  the  responsibility  of  managing  FDTE.  For 
all  practical  purposes,  however,  the  limit  of  this  management  has 
extended  to  the  inclusion  of  outline  test  plans  (OTP's)  in  the  Five- 
Year  Test  Program  (FYTP) , 

As  a  matter  of  reality,  the  management  could  not  exceed  this  point 
to  any  great  degree  since  OTEA  has  been  principally  Involved  in  the 
field  of  OT  since  it~4ras  organized,  as  it-should  be,  since  OT  is  its 
major  function. 

In  past  years,  there  have  been  a-number  of  studies  concerning  the 
redundancy  of  test  capability  at  MASSTER  and  CDEC.  As  a  result  of 
this  often-raised  point  of  inquiry,  the  two  agencies  have  come  to 
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develop  mission  statements  which  provide  justification  for  the 
continued  existence  of  both.  In  general  terms,  this  explanation  says 
that  MASSTER  conducts  tile  large-unit  subjective  teets  while  CISC 
conducts  the  snail-unit  objective  experiments.  While  the  justification 
is  plausible  in  many  respects  and  wliely  accepted  at  high  levels  of 
the  Arny,  nowhere  in  printed  regulations  or  directives  can  operational 
scopes  of  this  nature  be  found  for  the  two  organizaitons. 

MASSTER  receives  its  missions  froa  a  number  of  different  commands 
and  agencies.  It  is  at  the  sane  tine  a  subordinate  element  of  FORSOOM, 
commanded  *  the  III  Corps  Commander  at  Fort  Hood.  Although  FORSCOM 
has  no  responsibility  in  the  field  of  testing,  the  command  relation¬ 
ship  has  been  justified  on  the  grounds  that  troop  support  for  tests 
is  facilitated  by  this  arrangement.  Although  support  may  in  fact 
be  facilitated,  we  see  a  situation  where  a  unit  receives  all  of  its 
tasking  from  sources  other  than  its  higher  headquarters.  This  is  a 
violation  of  proven  managerial  procedure. 

In  order  to  give  direction  to  and  coordinate  this  divided  force 
test  responsibility,  both  CDEC  and  MASSTER  should  report  to  a  common 
agency  which  has  responsibility  for  all  Force  Testing.  The  agency 
should  have  no  other  responsibility  and  should  command  and  assign 
tasks  to  both  CDEC  and  MASSTER.  Since  the  schools  would  ideally 
initiate  tests,  the  agency  should  become  an  independent  tester  for 
TRADOC.  TRADOC  would,  therefore,  assume  responsibility  for  all  Army 
Force  Testing  through  this  force  test  and  evaluation  agency. 
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The  total  headquarters  strength  of  CDEC  and  MASS TER  exceeds 
1,300.  From  these  two  headquarters,  sufficient  spaces  could  be 
withdrawn  to  establish  the  propoeed  test  agency.  Thus,  no  increase 
in  Army  total  strength  would  result. 

RECOMMENDATION 

Assign  1RAD0C  responsibility  for  all  Any  Force  Testing  (FT). 
Form  a  Force  Test  and  Evaluation  Agency  (FTEA)  within  TRADOC  to  give 
direction  to  and  coordinate  FT.  Constitute  the  FTEA  from  currently 
authorized  spaces  at  CDEC  and  MASSTER.  Place  CDEC  and  MASSTER  under 
command  of  FTEA. 
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FINDING 

Interpretation  of  POD  Policy 

In  implementing  DOC  policy,  the  Army  hae  not  maintained  a  flexible 
attitude  toward  testing.  In  particular,  the  Army  has  over  structured 
its  Operational  Testing  procedures  and  overemphasized  its  organizational 
placement. 


Discussion 

As  previously  stated  in  Chapter  III,  Army  regulations  and  direc¬ 
tives  are  either  not  published  or  vague  in  intent.  Failure  to 
publish  either  AR  70-10  or  AR  71-3  for  more  than  two  years  has  added 
to  the  confusion.  These  draft  regulations  do  not  reflect  the  DOD 
concepts  of  OT  and  are  contradictory  in  requiring  both  the  use  of 
significant  scientific  techniques  and  testing  in  troop  field  exercises. 
Moreover,  there  are  many  Instances  when  "Independent  testing," 
"independent  evaluation,"  and  "separate  testing"  are  all  used  to  infer 
a  need  for  two  complete  and  independent  systems  for  design,  conduct, 
reporting,  and  evaluation  of  DTs  and  OTs .  The  basic  principles  in 
the  DOD  guidance  are  directed  toward  supplementary  testing  and 
independent  evaluations,  not  duplicative  testing. 

A  further  confusion  exists  in  the  amount  of  testing  required  for 
each  developmental  item.  Even  though  the  various  systems  under 
development  are  categorized  as  either  major,  selected  non-major  or 
other  non-major  systems  with  differing  levels  of  approval,  no  sucti 
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differentiation  is  aade  for  testing.  The  vagueness  of  the  draft 
regulations  is  such  that  all  phases  (I,  II,  and  III)  of  OT  and  DT 
could  be  expected  for  each  item  being  developed,  regardless  of  its 
relative  Importance. 

The  last  apparent  conflict  between  DOD  intent  and  DA  interpretation 

is  in  the  organizational  location  of  the  Any  testing  activities. 

For  example  DOD  Directive  500.3  states  that: 

Development  Test  and  Evaluation  is  planned, 
conducted,  and  monitored  by  the  developing 
agency  of  the  DOD  Component,  and  the  results 
thereof  are  reported  by  that  agency  to  the 
responsible  Military  Service  Chief. 

The  same  directive  further  states  that: 

In  each  DOD  Component  there  will  be  one  major 
field  agency,  separate  and  distinct  from  the 
developing/procuring  command  and  from  the  using 
command  which  will  be  responsible  for  Operational 
Test  and  Evaluation  and  which  will  report  the 
results  of  its  independent  test  and  evaluation 
directly  to  the  Military  Service  Chief. 

It  is  our  opinion  that  this  DOD  guidance  does  not  require  either  the 

developmental  tester  or  the  operational  tester  to  be  organizationally 

placed  directly  under  the  Office  of  the  Chief  of  Staff.  The  Amy, 

having  decentralized  the  level  of  decisionmaking  for  a  large  number 

'  of  its  materiel  developments,  has  in  many  cases  placed  the  operational 

tester  at  a  considerable  higher  command  echelon  than  the  approving 

authority  for  the  materiel  development.  The  location  of  OTEA  at  a 

lower  echelon  of  command  would  not  preclude  reporting  the  results 

of  OT  evaluations  directly  to  the  Chief  of  Staff  when  necessary. 
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BECOME  NDAT  ION 


First,  publish  AR  70-10  immediately  and  in  it  specify  that 
there  not  be  a  rigid  requirement  for  all  phase*’  of  OT.  Secondly, 
give  further  study  to  the  organizational  placement  of  OTEA. 
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The  Coordinated  Test  Program 

The  CTP  Is  not  being  properly  coordinated  in  order  to  injure  that 
(1)  the  right  Issues  are  tested  and  (2)  there  is  no  duplication  of 
testing  between  D7  and  OT. 

Regulations  covering  the  CTP  are  clear  as  to  intent  of  coordination. 
In  fact,  they  specify  that  everyone  possibly  concerned  with  the  materiel 
(the  developer,  the  tester,  the  logistician,  the  trainer,  and  so 
forth)  will  concur  in  the  CTP.  In  short,  the  CTP  has  the  making  of 
a  fine  management  tool;  however,  its  preparation,  coordination, 
distribution,  and  updating  are  the  responsibility  of  the  materiel 
developer  who  has  no  control  over  the  input  from  the  operational 
tester.  Although  there  are  detailed  requirements  for  each  involved 
agency  to  coordinate  with  all  others,  there  is  no  mechanism  for 
deciding  which  critical  issues  will  be-addressed,  or  which  tester 
(DT  or  OT)  will  conduct  what  test.  Since  testing  constitutes  a 
significant  part  of  the  overall  cost  of  development  and  time  for 
testing  is  extremely  limited,  it  would  appear  that  the  proper  division 
of  resources  and  testing  time  can  only  be  managed  at  a  higher  echelon. 

It  would  be  preferable  that  this  echelon  be  the  one  which  is  responsible 
for  the  entire  materiel  acquisition  process.  With  the  new  Army  Staff 
reorganization  such  a  staff  element  now  exists — the  DCSRDA. 
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RECOMMENDATION 


Deputy  Chief  of  Steff  Research,  Developaent 
have  the  overall  responsibility  for  the  aanage* 


and  Acquisition  should 
>nt  of  the  CTP. 
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FINDING 


Discussion 

Through  the  years,  OCRD  formed  rather  close  ties  with  the  materiel 
developer  and  as  a  result  was  called  upon  to  represent  the  developer's 
view  on  the  DA  staff.  This  position  of  DA  proponency  extended  to 
the  field  of  developmental  testing  as  well. 

In  recent  years  as  user  testing  came  to  the  forefront,  it  was 
believed  that  these  tests  also  needed  a  voice  on  the  DA  staff.  The 
result  was  the  appointment  of  ACSFOR  to  fill  the  role. 

While  user  tests  were  the  staff  responsibility  of  ACSFOR,  this 
function  was  largely  removed  from  the  coordinating  DA  staff  when 
ACSFOR  passed  much  of  its  user  test  management  to  its  subordinate 
field  operating  agency,  OTEA.  OTEA  has  been,  in  fact,  an  operator 
heavily  involved  in  the  execution  of  OT  and  thus  unable  to  effectively 
perform  the  DA  staff  coordinating  role. 

We,  therefore,  saw  the  DA  ACSFOR  staff  looking  to  OTEA  to  manage 
user  testing.  While  in  fact  OTEA  is  not  located  in  the  Pentagon  and 
further,  as  operators,  they  had  insufficient  time  to  handle  DA  staff 
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OT  coordination  utters  or  to  unage  FDTE.  Since  the  new  DA  staff 
organization  placet*  OTEA  directly  under  the  Chief  of  Staff  without 
an  intervening  staff  agency,  the  problem  will  be  further  aggravated. 

With  the  forutlon  of  OTEA,  there  appears  to  have  developed  a 
certain-unnecessary  spirit  of  confrontation  between  the  developmental 
and  operational  testers.  As  a  result  of  this  conflict,  agreement 
on  scope  of  operation  and  responsibilities  cannot  be  reached.  Thus, 
regulations  and  directives  which  are  most  important  to  test  and 
evaluation  remain  unpublished  or  badly  out  of  date. 

Since  we  now  have  a  staff  department  responsible  for  both  research 
and  development  as  well  as  acquisition,  (DCSBDA) ,  it  would  appear 
that  all  testing  related  to  the  acquisition  process  should  be  the 
responsibility  of  that  staff  department.  The  result  of  assigning 
DCSRDA  staff  responsibility  for  both  OT  and  DT  would  go  far  toward 
alleviating  the  current  difficulties.  The  DCSRDA  would  then  be  able 
to  provide  policy  guidance  which  would  enhance  operations  in  those 
areas  where  DT  and  OT  tend  to  overlap. 

Currently,  DA  staff  agencies  responsible  for  developmental  testing 
display  little  interest  in  operational  tests  and  the  reverse  is 
also  true.  Although  some  support  this  partisan  test  advocacy,  there 
appears  to  be  a  crying  need  for  a  strong  DA  staff  "honest  broker" 
in  the  field  of  acquisition  testing.  DCSRDA  could  well  perform  that 
role. 
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It  would  be  advantageous  for  all  testing  to  cone  under  one  DA 
staff  department;  however,  DCSRDA  has  no  compelling  reason  to  become 
Involved  In  Force  Testing.  These  tests  of  doctrine,  organization, 
and  tactics  are  far  more  compatible  with  the  funcitons  of  DCSOPS. 

The  managment  procedures  outlined  in  AR  71-3  (draft)  describing 
the  FYTP  and  the  TSARC  should  be  continued  for  acquisition  testing 
(DT  and  OT)  with  the  exception  of  chairmanship  transfer  from  OTEA 
to  DCSRDA.  In  addition,  the  DCSOPS  should  establish  a  FYTP  and  TSARC 
for  Force  Testing.  DCSOPS  should  chair  this  second  TSARC. 

RECOMMENDATION 

Assign  DCSRDA  DA  staff  responsibility  for  both  DT  and  OT,  thus 
centralizing  policy  guidance  for  all  acquisition  testing.  Assign 
DCSOPS  DA  staff  responsibility  for  all  Force  Testing. 


1^8. 
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Fragmentation  of  OT 

Operational  teat  execution  responsibility  within  the  Army  is 
currently  divided  between  OTEA  and  TRADOC.  OTEA  conducts  OT  for  all 
material  with  high  visibility ,  while  all  remaining  systems  are 
forwarded  to  TRADOC.  The  TECOM  Boards  currently  perform  Service 
testing  for  the  developer.  This  Service  testing  is  basically  a 
preliminary  operational  test  performed  by  the  developer. 

Discussion 

As  we  have  seen,  OTEA  plans,  directs  and  evaluates  OT  for  all 
major  and  selected  non-major  systems  while  it  coordinates  OT  for  all 
other  non-major  systems.  TRADOC  directs  other  non-major  OT  which 
is  actually  conducted  by  the  functional  centers  or  schools. 

Through  this  delegation  of  responsibility,  OTEA  is  able  to 
direct  the  attention  of  its  small  staff  to  those  few  acquisition 
systems  of  primary  importance  while  managing  by  exception  those  tests 
conducted  of  other  non-major  systems. 

By  this  management  system,  however,  OT  responsibility  has  been 
divided  between  two  agencies .  This  naturally  allows  for  varying 
policy  and  direction.  In  addition  a  system  might  begin  the 
acquisition  process  with  only  limited  high-level  interest  and  thus 
O'1'  would  be  executed  by  TRADOC.  For  numerous  reasons,  interest 
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could  increase,  causing  redesignation  of  the  system  to  major  or 
selected  non-major,  and  thus  requiring  a  transfer  of  OT  responsibility. 
Therefore,  the  vesting  of  responsibility  for  all  OT  in  one  agency 
would  be  desirable  providing  that  organization  had  adequate  resources 
to  accomplish  the  mission  and  still  not  dilute  the  attention  given 
to  critical  systems. 

As  has  been  pointed  out  earlier,  one  of  the  major  disagreements 
between  the  Operational  Tester  and  the  Developer  is  the  point  of 
Service  Test  conclusion  and  Operational  Test  beginning.  TECOM 
currently  possesses  a  sizeable  Operational /Service  Testing 
capability  within  its  boards.  At  tbe  same  time,  it  is  generally 
agreed  that  although  TECOM  should  do  limited  "soldier-proofing," 
it  appears  that  they  are  currently  encroaching  into  the  operational 
testing  field  with  their  Service  tests. 

The  assignment  of  the  TECOM  Test  Boards  to  OTEA  would  substantially 
increase  OTEA's  capability  and  allow  them  to  assume  full  responsibility 
for  OT.  TRADOC  would  then  be  removed  from  the  material  acquisition 
testing,  where  it  has  limited  reason  for  involvement.  TRADOC  could 
then  direct  its  attention  to  the  neglected  Force  Development  Testing, 
an  area  of  appropriate  and  critical  interest. 

Testing  of  major  and  selected  non-major  systems  could  continue 
as  currently  executed  by  OTEA;  however,  OTEA  with  the  boards  available 
could  assume  responsibility  for  all  OT.  The  boards  would  principally 
conduct  nonselected  OT  but  would  also  provide  OTEA  with  added  expertise 
in  tests  of  major  systems. 
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Not  only  would  the  reassignment  of  the  Boards  to  OTEA  provide 
opportunity  for  consolidation  of  OT,  but  it  would  also  defuse  the 
argument  between  the  developmental  and  the  operational  testers  as 
to  the  extent  of  Se.*vice  testing  in  DT.  The  Service  test  capability, 
now  available  to  TECO'  ,  would  be  given  to  the  operational  tester. 

RECOMMENDATION 

Assign  the  TECOM  Boards  to  OTEA  for  operational  testing  purposes, 
and  have  OTEA  assume  full  responsibility  for  all  OT. 
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FINDING 


Troop  Involvement  in  DT 

There  la  confusion  concerning  the  amount  of  troop  involvement 
required  in  Development  Testing. 

Discussion 

The  Service  phase  of  DT,  as  presently  conducted  by  TECOM,  is 
duplicative  in  nature  to  OT.  Although  TECOM  views  the  service  phase 
as  answering  questions  about  the  materiel  while  OT  is  to  answer 
questions  which  are  raised  because  new  materiel  is  being  introduced, 
this  definition  does  not  agree  with  the  DOD  guidance.  In  fact,  the 
DOD  requirement  to  accomplish  OT  with  repreesentative  user  troops  in  a 
realistic  environment  is  similar  to  the  specified  criteria  for  the 
service  phase.  At  the  same  time,  there  is  a  definite  requirement 
for  the  developer  to  have  some  user  troops  assist  him  during  the 
development  of  materiel,  particularly  in  trouble-shooting  equipment. 

In  the  environment  of  a  field  test  such  as  that  used  in  Operational 
Testing  there  will  be  frequent  occasions  when  it  is  almost  impossible 
to  determine  the  origin  of  a  problem,  that  is  to  say  whether  the 
problem  is  materiel-oriented  or  soldier-oriented.  This  is  something 
that  the  developer  must  know  as  the  materiel  proceeds  through  the 
evolution  of  development  and  testing. 

Although  current  directives,  both  those  published  and  those  in 
draft,  call  for  the  developer  to  conduct  "soldier-proofing"  the  term 
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is-ill  defined.  To  assist  in  the  design  and  redesign  of  equipment 

AMC  suet  have  a  Halted  number  of  user  troops  available  and  oust 

have  a  Halted  capability  to  "soldier-proof."  In  order  to  preclude 

further  duplication  or  confusion  "soldier-proofing-  rust  be  better 

defined  and  understood.  Since  the  ten  is  poorly  understood, 

another  tern,  "soldler-aateriel  coapatiblUty  testing,"  nay  be  clearer. 

A  proposed  definition  of  this  ten  is: 

Soldier-aateriel  compatibility  testing  is  that 
portion  of  development  testing  (DT,  I,  II,  III) 
through  which  the  developer  verifies  that  the 
tested  materiel  has  met  design  requirements  and 
will  attain  the  required  technical  performance 
characteristics  when  operated  and  maintained  by 
user  personnel.  Such  testing  is  performed  on 
prototype  models  and  production  items,  their 
component  parts  and,  where  required,  the 
complementary  interface  of  government-furnished 
equipment  and  demonstrates  to  the  developer 
that  the  materiel  will  perform  safely  in  the 
hands  of  user  personnel.  Test  materiel  includes 
both  hardware  and  software  (e.g.,  manuals)  that 
the  operator  and  maintainer  will  use.  Personnel 
required  for  such  testing  are  soldiers  with 
about  the  same  skills,  training,  and  field 
experience  that  would  be  available  in  a  troop 
unit  equipped  with  that  materiel.  In  other 
words,  soldier-materiel  compatibility  testing 
relates  to  the  ability  to  use  a  piece  of  equip¬ 
ment,  rather  than  to  tactical  utility  of  the 
equipment  itself.  It  is  not  the  test  of  equip¬ 
ment  in  its  operational  environment. 


RECOMMENDATION 


AMC  retain  a  limited  capability  to  do  soldier-materiel  com¬ 
patibility  testing,  as  defined  above. 

RECOMMENDATION 

Adopt  the  proposed  definition  for  soldier-materiel  compatibility 
testing. 
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